Firefox OS/Performance/Investigating Alerts: Difference between revisions
| Line 20: | Line 20: | ||
The score can map directly to the confidence of the alert sustaining over time. The range is from 2-4 where 4 is .9999% sure. In the above example, we have a score of 3.32 which is .99932% confident this is a sustained regression. | The score can map directly to the confidence of the alert sustaining over time. The range is from 2-4 where 4 is .9999% sure. In the above example, we have a score of 3.32 which is .99932% confident this is a sustained regression. | ||
The next section gives you direct links to the code that built B2G used for this test. | |||
[[File:Alert_gaia_current.png|200px]] | |||
The gaia revision is a link to github that shows the specific changes to gaia | |||
[[File:Alert_gecko_current.png|200px]] | |||
The gecko revision is a link to github showing the current revision of gecko that we are using | |||
[[File:Alert_csets_previous.png|200px]] | |||
The links to previous are links to gaia/gecko revisions for the previous build that ran tests. | |||
Revision as of 18:02, 14 May 2014
When you get an email from fxos-perf-alerts@mozilla.com, there is a good chance it is an automated email.
We have two types of automated alerts:
- Ingestion Alerts (we were expecting data and never received it)
- Regression Alerts (the values of the reported data have changed)
Regression Alerts
A regression alert has a Title in this format
'[ALERT][B2G] <testname> regressed by <-x.x> in <suite>'
example
'[ALERT][B2G] email_memory regressed by -3.6 in vsize, vsize'
The first part of the Alert is the score:
The score can map directly to the confidence of the alert sustaining over time. The range is from 2-4 where 4 is .9999% sure. In the above example, we have a score of 3.32 which is .99932% confident this is a sustained regression.
The next section gives you direct links to the code that built B2G used for this test.
The gaia revision is a link to github that shows the specific changes to gaia
The gecko revision is a link to github showing the current revision of gecko that we are using
The links to previous are links to gaia/gecko revisions for the previous build that ran tests.