TestEngineering/Performance/Sheriffing/Alerts

From MozillaWiki
< TestEngineering‎ | Performance‎ | Sheriffing
Revision as of 09:06, 16 January 2018 by Igoldan (talk | contribs) (platform_microbench description)
Jump to navigation Jump to search

Perfherder alerts

General triage process

<how triaging is similar among all alerts>

Types of alerts

Talos

  • short description: <provide it> link
  • frequency: daily, little more than a dozen alerts
  • coalesced by SETA?: yes (often requires backfilling)
  • available on platforms:
    • Windows: 7 32bit, 10 64bit (OPT, PGO builds)
    • Linux: 64bit (OPT, PGO builds)
    • OS X: 10.10 (OPT builds only)
  • triaging specifics:

build_metrics

  • short description: Monitor build times on multiple platforms, the size of the installers and other compiler-specific insights.
  • frequency: little less than daily, around 5 alerts
  • contact: :froydnj, :ted.mielczarek, :gps
  • coalesced by SETA?: no (doesn't require backfilling)
  • available on platforms:
    • Windows: 32/64bit (OPT, No-OPT, Mingw builds)
    • Linux: 32/64bit (OPT, No-OPT builds)
    • OS X: 10.10 (cross, no-cross builds)
    • Android: 4.0, 4.2, 5.0
  • triaging specifics:
    • often easy to investigate
    • most alerts aren't noisy
    • when investigating, one should look for build config changes <ask :gps to provide more data>
    • build times often spike upwards for just a short time; they then lower to previous levels thanks to caching mechanisms set in place. We mark these as invalid alerts.

Autophone

AWSY

  • short description: <provide it> link
  • frequency: every 2-3 days, around half a dozen
  • contact: :erahm
  • coalesced by SETA?: yes
  • available on platforms:
    • Windows: 32/64bit (OPT, PGO builds)
    • Linux: 64bit (OPT builds)
    • OS X: 10.10 (OPT builds)
    • Android: 4.2, 4.3 (OPT builds)
  • triaging specifics:
    • retriggering/backfilling takes some time (>1h per test), so one must not abuse in collecting missing graph data

platform_microbench

  • short description: <provide it>
  • frequency: daily, around 1-2 dozen alerts
  • contact:
  • coalesced by SETA?: yes
  • available on platforms:
    • Linux: 32bit (OPT builds), 64bit (OPT, PGO, ASAN builds)
    • Windows: 7 32bit, 10 64bit (OPT builds)
    • OS X: 10.10 (OPT builds)
  • triaging specifics:
    • happen very often; unless triaged, they quickly pile up
    • very noisy alerts; often many of the alerts turn out to be invalid
    • cheap to retrigger, as each test takes <20min to finish; still, one should not abuse this