947
edits
Auto-tools/Projects/Signal From Noise/Execution2012: Difference between revisions
Jump to navigation
Jump to search
Auto-tools/Projects/Signal From Noise/Execution2012 (view source)
Revision as of 22:55, 28 December 2012
, 28 December 2012→Goals of Signal from Noise
(→Signal From Noise: rewording) |
|||
| Line 3: | Line 3: | ||
Signal from Noise builds on top of the work of Larres (https://wiki.mozilla.org/images/c/c0/Larres-thesis.pdf ) and Stephen Lewchuk who analyzed the statistics as yielded by Talos and suggested areas for improvement. | Signal from Noise builds on top of the work of Larres (https://wiki.mozilla.org/images/c/c0/Larres-thesis.pdf ) and Stephen Lewchuk who analyzed the statistics as yielded by Talos and suggested areas for improvement. | ||
== Goals of Signal from Noise == | == Goals of Signal from Noise == | ||
* Enhance fidelity of performance testing measurements: we want to ensure that (nearly) all regressions are detected. We also want to ensure that as few false positives (the noise) are reported as possible | |||
* Enhance fidelity of performance testing measurements: we want to ensure that (nearly) all regressions and improvements are detected. We also want to ensure that as few false positives (the noise) are reported as possible | |||
* Understand our performance statistics models and assumptions end-to-end and ensure that the statistics we are using are valid | * Understand our performance statistics models and assumptions end-to-end and ensure that the statistics we are using are valid | ||
** Document and expose these statistics in such a way that they are transparent | ** Document and expose these statistics in such a way that they are transparent | ||