Auto-tools/Projects/Signal From Noise/Meetings/2013-01-16: Difference between revisions
< Auto-tools | Projects | Signal From Noise | Meetings
(→Talos: bug 816634) |
|||
| (7 intermediate revisions by 3 users not shown) | |||
| Line 1: | Line 1: | ||
= Datazilla = | = Datazilla = | ||
* Updated https://datazilla.readthedocs.org/en/latest/ | |||
* Added in-application help | |||
* Fixed a cross browser UI layout bug | |||
* Added the threshold revision for each metric datum. In the UI this is referred to as the "compared to" field which allows a user to immediately see what revision's replicates were used in the t-test. It also helps explain the pattern of the "No Fit" visualization of the trend line. | |||
* In process of testing all of these changes in staging, should be in production today https://bugzilla.mozilla.org/show_bug.cgi?id=831515. | |||
= Talos = | = Talos = | ||
| Line 6: | Line 10: | ||
* talos goes orange when we crash- now we can easily go orange when a test fails!! | * talos goes orange when we crash- now we can easily go orange when a test fails!! | ||
* talos now generates new-style TBPL output: {{bug|816634}}; this still has be be parsed from TBPL and possibly fine-tuned | * talos now generates new-style TBPL output: {{bug|816634}}; this still has be be parsed from TBPL and possibly fine-tuned | ||
* will probably need (at least one) new release soon (what else is new?) | |||
= Round Table = | = Round Table = | ||
* BenWa has proposed adding thresholds per test to the tree, everything outside the threshold would be orange and require a perf sherriff to adjust the value. | * BenWa has proposed adding thresholds per test to the tree, everything outside the threshold would be orange and require a perf sherriff to adjust the value. | ||
= Action Items = | |||
* [all] track the hanging ingestion job | |||
* [all, future] come up with a solution for retriggers | |||
* [jeads] outline possible corner cases where we need a solution (i.e. thresholds) | |||
** [jmaher, all] work with perf team to solve this, ideally per page | |||
* [jeads] hack into datazilla: if (osx) branchname += "-Non-PGO" | |||
* [jmaher] post new talos with datazilla friendlier posting and missing 'talos ' in test name | |||
* [all] verify tp5n is accurate | |||
* [all, future] start verifying other data, tsvg* | |||
** future data to verify would be ts* and counters | |||
* [jhammel] add datazillla links to TBPL | |||
* [all] turn jobs orange for tp5n | |||
Latest revision as of 20:03, 17 January 2013
Datazilla
- Updated https://datazilla.readthedocs.org/en/latest/
- Added in-application help
- Fixed a cross browser UI layout bug
- Added the threshold revision for each metric datum. In the UI this is referred to as the "compared to" field which allows a user to immediately see what revision's replicates were used in the t-test. It also helps explain the pattern of the "No Fit" visualization of the trend line.
- In process of testing all of these changes in staging, should be in production today https://bugzilla.mozilla.org/show_bug.cgi?id=831515.
Talos
- lots of updates to crash reporting
- talos goes orange when we crash- now we can easily go orange when a test fails!!
- talos now generates new-style TBPL output: bug 816634; this still has be be parsed from TBPL and possibly fine-tuned
- will probably need (at least one) new release soon (what else is new?)
Round Table
- BenWa has proposed adding thresholds per test to the tree, everything outside the threshold would be orange and require a perf sherriff to adjust the value.
Action Items
- [all] track the hanging ingestion job
- [all, future] come up with a solution for retriggers
- [jeads] outline possible corner cases where we need a solution (i.e. thresholds)
- [jmaher, all] work with perf team to solve this, ideally per page
- [jeads] hack into datazilla: if (osx) branchname += "-Non-PGO"
- [jmaher] post new talos with datazilla friendlier posting and missing 'talos ' in test name
- [all] verify tp5n is accurate
- [all, future] start verifying other data, tsvg*
- future data to verify would be ts* and counters
- [jhammel] add datazillla links to TBPL
- [all] turn jobs orange for tp5n