Firefox OS/Performance/Investigating Alerts: Difference between revisions

Jump to navigation Jump to search
m
Lakrits moved page FirefoxOS/Performance/Investigating Alerts to Firefox OS/Performance/Investigating Alerts: The official spelling of "Firefox OS" leaves a space between the two parts of the name. It's easier to find a page if the spelling of...
m (Lakrits moved page FirefoxOS/Performance/Investigating Alerts to Firefox OS/Performance/Investigating Alerts: The official spelling of "Firefox OS" leaves a space between the two parts of the name. It's easier to find a page if the spelling of...)
 
(2 intermediate revisions by 2 users not shown)
Line 20: Line 20:
[[File:Alert_score.png|200px]]
[[File:Alert_score.png|200px]]


The score can map directly to the confidence of the alert sustaining over time.  The range is from 2-4 where 4 is .9999% sure.  In the above example, we have a score of 3.32 which is .99932% confident this is a sustained regression.
This score is meant to be abstract representation of how important this alert is compared to the othersThere are many factors that go into generating a score: 
* The p-value of the test used,
* a multiplier for the type of test (compensating for it's propensity to send false alerts),
* the relative size of the regression, and
* in the case of compound alerts; a mixture of the component alert scores.
 
Right now, The score currently maps to the p-value of the median test used to detect the regression: <code>score = -log10(p-value)</code>. It is currently set to alert when score > 2 (p-value < 0.01), and does not go above 4 (p-value > 0.0001) given the current window size.


The next section gives you direct links to the code that built B2G used for this test.
The next section gives you direct links to the code that built B2G used for this test.
Line 31: Line 37:


The gecko revision is a link to github showing the current revision of gecko that we are using. {{bug|1010384}} tracks getting these links to work.
The gecko revision is a link to github showing the current revision of gecko that we are using. {{bug|1010384}} tracks getting these links to work.


== Detailed Regressions ==
== Detailed Regressions ==
Line 62: Line 67:


=== Datazilla links ===
=== Datazilla links ===
The next section is the Datazilla links, these are mostly self explanatory and for more information on what the view in Datazilla means check out [[/Bisecting_Regressions|Bisecting_Regressions]]:
The next section is the Datazilla links, these are mostly self explanatory and for more information on what the view in Datazilla means, check out [[../Bisecting_Regressions|Bisecting Regressions]]:
 
[[File:Alert_datazilla_link.png|600px]]
[[File:Alert_datazilla_link.png|600px]]


3,860

edits

Navigation menu