Revision as of 19:56, 17 January 2013

Datazilla

Updated https://datazilla.readthedocs.org/en/latest/
Added in-application help
Fixed a cross browser UI layout bug
Added the threshold revision for each metric datum. In the UI this is referred to as the "compared to" field which allows a user to immediately see what revision's replicates were used in the t-test. It also helps explain the pattern of the "No Fit" visualization of the trend line.
In process of testing all of these changes in staging, should be in production today https://bugzilla.mozilla.org/show_bug.cgi?id=831515.

lots of updates to crash reporting
talos goes orange when we crash- now we can easily go orange when a test fails!!
talos now generates new-style TBPL output: bug 816634; this still has be be parsed from TBPL and possibly fine-tuned
will probably need (at least one) new release soon (what else is new?)

BenWa has proposed adding thresholds per test to the tree, everything outside the threshold would be orange and require a perf sherriff to adjust the value.

track the hanging ingestion job
come up with a solution for retriggers
outline possible corner cases where we need a solution (i.e. thresholds)
- work with perf team to solve this, ideally per page
hack into datazilla: if (osx) branchname += "-Non-PGO"
post new talos with datazilla friendlier posting and missing 'talos ' in test name
verify tp5n is accurate
start verifying other data, tsvg*
future data to verify would be ts* and counters
add datazillla links to TBPL

@@ Line 25: / Line 25: @@
 * start verifying other data, tsvg*
 * future data to verify would be ts* and counters
+* add datazillla links to TBPL