Auto-tools/Projects/Signal From Noise/Meetings/2013-01-16: Difference between revisions
< Auto-tools | Projects | Signal From Noise | Meetings
Jump to navigation
Jump to search
| Line 14: | Line 14: | ||
= Round Table = | = Round Table = | ||
* BenWa has proposed adding thresholds per test to the tree, everything outside the threshold would be orange and require a perf sherriff to adjust the value. | * BenWa has proposed adding thresholds per test to the tree, everything outside the threshold would be orange and require a perf sherriff to adjust the value. | ||
= Action Items = | |||
* track the hanging ingestion job | |||
* come up with a solution for retriggers | |||
* outline possible corner cases where we need a solution (i.e. thresholds) | |||
** work with perf team to solve this, ideally per page | |||
* hack into datazilla: if (osx) branchname += "-Non-PGO" | |||
* post new talos with datazilla friendlier posting and missing 'talos ' in test name | |||
* verify tp5n is accurate | |||
* start verifying other data, tsvg* | |||
* future data to verify would be ts* and counters | |||
Revision as of 19:54, 17 January 2013
Datazilla
- Updated https://datazilla.readthedocs.org/en/latest/
- Added in-application help
- Fixed a cross browser UI layout bug
- Added the threshold revision for each metric datum. In the UI this is referred to as the "compared to" field which allows a user to immediately see what revision's replicates were used in the t-test. It also helps explain the pattern of the "No Fit" visualization of the trend line.
- In process of testing all of these changes in staging, should be in production today https://bugzilla.mozilla.org/show_bug.cgi?id=831515.
Talos
- lots of updates to crash reporting
- talos goes orange when we crash- now we can easily go orange when a test fails!!
- talos now generates new-style TBPL output: bug 816634; this still has be be parsed from TBPL and possibly fine-tuned
- will probably need (at least one) new release soon (what else is new?)
Round Table
- BenWa has proposed adding thresholds per test to the tree, everything outside the threshold would be orange and require a perf sherriff to adjust the value.
Action Items
- track the hanging ingestion job
- come up with a solution for retriggers
- outline possible corner cases where we need a solution (i.e. thresholds)
- work with perf team to solve this, ideally per page
- hack into datazilla: if (osx) branchname += "-Non-PGO"
- post new talos with datazilla friendlier posting and missing 'talos ' in test name
- verify tp5n is accurate
- start verifying other data, tsvg*
- future data to verify would be ts* and counters