MozillaWiki

TestEngineering/Performance/Sheriffing/Alerts: Difference between revisions

Page Discussion

TestEngineering/Performance/Sheriffing/Alerts (view source)

Revision as of 13:59, 29 January 2020

781 bytes removed ,  29 January 2020
no edit summary
(provide some short descriptions)
No edit summary
 
(4 intermediate revisions by 2 users not shown)
Line 7: Line 7:


=== Talos ===
=== Talos ===
* short description: Monitors various time-based performance KPIs on the browser ([https://wiki.mozilla.org/index.php?title=Buildbot/Talos/Tests more details])
* short description: Monitors various time-based performance KPIs on the browser ([https://wiki.mozilla.org/index.php?title=Performance_sheriffing/Talos/Tests more details])
* frequency: daily, little more than a dozen alerts
* frequency: daily, little more than a dozen alerts
* [https://wiki.mozilla.org/Buildbot/Talos/Sheriffing/Tree_FAQ#What_is_coalescing coalesced] by SETA?: yes (often requires backfilling)
* [https://wiki.mozilla.org/Performance_sheriffing/Talos/Sheriffing/Tree_FAQ#What_is_coalescing coalesced] by SETA?: yes (often requires backfilling)
* available on platforms:
* available on platforms:
** Windows: 7 32bit, 10 64bit (OPT, PGO builds)
** Windows: 7 32bit, 10 64bit (OPT, PGO builds)
Line 15: Line 15:
** OS X: 10.10 (OPT builds only)
** OS X: 10.10 (OPT builds only)
* triaging specifics:
* triaging specifics:
=== build_metrics ===
=== build_metrics ===
* short description: Monitor build times on multiple platforms, the size of the installers and other compiler-specific insights.
* short description: Monitor build times on multiple platforms, the size of the installers and other compiler-specific insights.
Line 30: Line 31:
** when investigating, one should look for build config changes <ask :gps to provide more data>
** when investigating, one should look for build config changes <ask :gps to provide more data>
** build times often spike upwards for just a short time; they then lower to previous levels thanks to caching mechanisms set in place. We mark these as invalid alerts.
** build times often spike upwards for just a short time; they then lower to previous levels thanks to caching mechanisms set in place. We mark these as invalid alerts.
=== Autophone ===
 
* short description: Monitors mobile performance of browser ([https://wiki.mozilla.org/EngineeringProductivity/Autophone more details])
* frequency: every week or so, around 4 alerts
* contact: :bc:
* coalesced by SETA?: no
* available on platforms:
** Android: 4.2, 4.4, 6.0, 7.1
* triaging specifics:
** when investigating, one should look for Android related changes
** many of these tests are pretty noisy; often, they turn out to be invalid (one reason is devices overheat, which affects tests)
** should consider to needinfo? :bc:, to check status of suspect phone devices
** tricky to investigate; consider using also [http://phonedash.mozilla.org/ Phonedash], as it's a more precise investigation tool for mobile regressions
** retriggers are almost always needed, but results show up after a day or so
=== AWSY ===
=== AWSY ===
* short description: Monitors memory consumption by the browser([https://wiki.mozilla.org/AWSY/Tests more details])
* short description: Are We Slim Yet monitors memory consumption by the browser ([https://wiki.mozilla.org/AWSY/Tests more details])
* frequency: every 2-3 days, around half a dozen
* frequency: every 2-3 days, around half a dozen
* contact: :erahm
* contact: :erahm
Line 55: Line 44:
* triaging specifics:
* triaging specifics:
** retriggering/backfilling takes some time (>1h per test), so one must not abuse in collecting missing graph data
** retriggering/backfilling takes some time (>1h per test), so one must not abuse in collecting missing graph data
=== platform_microbench ===
=== platform_microbench ===
* short description: N/A
* short description: N/A
Davehunt
Confirmed users
2,177

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/1186826...1222980"