TestEngineering/Performance/Sheriffing/Alerts: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(awsy info)
(platform_microbench description)
Line 7: Line 7:


=== Talos ===
=== Talos ===
* short description: [https://wiki.mozilla.org/index.php?title=Buildbot/Talos/Tests link]
* short description: <provide it> [https://wiki.mozilla.org/index.php?title=Buildbot/Talos/Tests link]
* frequency: daily, little more than a dozen alerts
* frequency: daily, little more than a dozen alerts
* [https://wiki.mozilla.org/Buildbot/Talos/Sheriffing/Tree_FAQ#What_is_coalescing coalesced] by SETA?: yes (often requires backfilling)
* [https://wiki.mozilla.org/Buildbot/Talos/Sheriffing/Tree_FAQ#What_is_coalescing coalesced] by SETA?: yes (often requires backfilling)
* available on platforms:
* available on platforms:
** Windows: 7 32bit, 10 64bit (OPT, PGO builds)
** Linux: 64bit (OPT, PGO builds)
** Linux: 64bit (OPT, PGO builds)
** Windows: 7 32bit, 10 64bit (OPT, PGO builds)
** OS X: 10.10 (OPT builds only)
** OS X: 10.10 (OPT builds only)
* triaging specifics:
* triaging specifics:
Line 21: Line 21:
* [https://wiki.mozilla.org/Buildbot/Talos/Sheriffing/Tree_FAQ#What_is_coalescing coalesced] by SETA?: no (doesn't require backfilling)
* [https://wiki.mozilla.org/Buildbot/Talos/Sheriffing/Tree_FAQ#What_is_coalescing coalesced] by SETA?: no (doesn't require backfilling)
* available on platforms:
* available on platforms:
** Windows: 32/64bit (OPT, No-OPT, Mingw builds)
** Linux: 32/64bit (OPT, No-OPT builds)
** Linux: 32/64bit (OPT, No-OPT builds)
** Windows: 32/64bit (OPT, No-OPT, Mingw builds)
** OS X: 10.10 (cross, no-cross builds)
** OS X: 10.10 (cross, no-cross builds)
** Android: 4.0, 4.2, 5.0
** Android: 4.0, 4.2, 5.0
Line 32: Line 32:
=== Autophone ===
=== Autophone ===
=== AWSY ===
=== AWSY ===
* short description: [https://wiki.mozilla.org/AWSY/Tests link]
* short description: <provide it> [https://wiki.mozilla.org/AWSY/Tests link]
* frequency: every 2-3 days, around half a dozen
* frequency: every 2-3 days, around half a dozen
* contact: :erahm
* contact: :erahm
* [https://wiki.mozilla.org/Buildbot/Talos/Sheriffing/Tree_FAQ#What_is_coalescing coalesced] by SETA?: yes
* [https://wiki.mozilla.org/Buildbot/Talos/Sheriffing/Tree_FAQ#What_is_coalescing coalesced] by SETA?: yes
* available on platforms:
* available on platforms:
** Windows: 32/64bit (OPT, PGO builds)
** Linux: 64bit (OPT builds)
** Linux: 64bit (OPT builds)
** Windows: 32/64bit (OPT, PGO builds)
** OS X: 10.10 (OPT builds)
** OS X: 10.10 (OPT builds)
** Android: 4.2, 4.3 (OPT builds)
** Android: 4.2, 4.3 (OPT builds)
Line 44: Line 44:
** retriggering/backfilling takes some time (>1h per test), so one must not abuse in collecting missing graph data
** retriggering/backfilling takes some time (>1h per test), so one must not abuse in collecting missing graph data
=== platform_microbench ===
=== platform_microbench ===
* short description: <provide it>
* frequency: daily, around 1-2 dozen alerts
* contact:
* [https://wiki.mozilla.org/Buildbot/Talos/Sheriffing/Tree_FAQ#What_is_coalescing coalesced] by SETA?: yes
* available on platforms:
** Linux: 32bit (OPT builds), 64bit (OPT, PGO, ASAN builds)
** Windows: 7 32bit, 10 64bit (OPT builds)
** OS X: 10.10 (OPT builds)
* triaging specifics:
** happen very often; unless triaged, they quickly pile up
** very noisy alerts; often many of the alerts turn out to be invalid
** cheap to retrigger, as each test takes <20min to finish; still, one should not abuse this

Revision as of 09:06, 16 January 2018

Perfherder alerts

General triage process

<how triaging is similar among all alerts>

Types of alerts

Talos

  • short description: <provide it> link
  • frequency: daily, little more than a dozen alerts
  • coalesced by SETA?: yes (often requires backfilling)
  • available on platforms:
    • Windows: 7 32bit, 10 64bit (OPT, PGO builds)
    • Linux: 64bit (OPT, PGO builds)
    • OS X: 10.10 (OPT builds only)
  • triaging specifics:

build_metrics

  • short description: Monitor build times on multiple platforms, the size of the installers and other compiler-specific insights.
  • frequency: little less than daily, around 5 alerts
  • contact: :froydnj, :ted.mielczarek, :gps
  • coalesced by SETA?: no (doesn't require backfilling)
  • available on platforms:
    • Windows: 32/64bit (OPT, No-OPT, Mingw builds)
    • Linux: 32/64bit (OPT, No-OPT builds)
    • OS X: 10.10 (cross, no-cross builds)
    • Android: 4.0, 4.2, 5.0
  • triaging specifics:
    • often easy to investigate
    • most alerts aren't noisy
    • when investigating, one should look for build config changes <ask :gps to provide more data>
    • build times often spike upwards for just a short time; they then lower to previous levels thanks to caching mechanisms set in place. We mark these as invalid alerts.

Autophone

AWSY

  • short description: <provide it> link
  • frequency: every 2-3 days, around half a dozen
  • contact: :erahm
  • coalesced by SETA?: yes
  • available on platforms:
    • Windows: 32/64bit (OPT, PGO builds)
    • Linux: 64bit (OPT builds)
    • OS X: 10.10 (OPT builds)
    • Android: 4.2, 4.3 (OPT builds)
  • triaging specifics:
    • retriggering/backfilling takes some time (>1h per test), so one must not abuse in collecting missing graph data

platform_microbench

  • short description: <provide it>
  • frequency: daily, around 1-2 dozen alerts
  • contact:
  • coalesced by SETA?: yes
  • available on platforms:
    • Linux: 32bit (OPT builds), 64bit (OPT, PGO, ASAN builds)
    • Windows: 7 32bit, 10 64bit (OPT builds)
    • OS X: 10.10 (OPT builds)
  • triaging specifics:
    • happen very often; unless triaged, they quickly pile up
    • very noisy alerts; often many of the alerts turn out to be invalid
    • cheap to retrigger, as each test takes <20min to finish; still, one should not abuse this
Retrieved from "https://wiki.allizom.org/index.php?title=TestEngineering/Performance/Sheriffing/Alerts&oldid=1186818"