Firefox OS/Performance/Investigating Alerts: Difference between revisions

Firefox OS/Performance/Investigating Alerts (view source)

453 bytes added , 25 June 2014

Confirmed users

513

edits

@@ Line 20: / Line 20: @@
 [[File:Alert_score.png|200px]]
-The score can map directly to the confidence of the alert sustaining over time.  The range is from 2-4 where 4 is .9999% sure.  In the above example, we have a score of 3.32 which is .99932% confident this is a sustained regression.
+This score is meant to be abstract representation of how important this alert is compared to the others.  There are many factors that go into generating a score:
+* The p-value of the test used,
+* a multiplier for the type of test (compensating for it's propensity to send false alerts),
+* the relative size of the regression, and
+* in the case of compound alerts; a mixture of the component alert scores.
+Right now, The score currently maps to the p-value of the median test used to detect the regression: <code>score = -log10(p-value)</code>.  It is currently set to alert when score > 2 (p-value < 0.01), and does not go above 4 (p-value > 0.0001) given the current window size.
 The next section gives you direct links to the code that built B2G used for this test.
@@ Line 31: / Line 37: @@
 The gecko revision is a link to github showing the current revision of gecko that we are using. {{bug|1010384}} tracks getting these links to work.
 == Detailed Regressions ==