TestEngineering/Performance/Sheriffing/Alerts

Perfherder alerts

General triage process

<how triaging is similar among all alerts>

Types of alerts

Talos

• short description: link
• frequency: daily, little more than a dozen alerts
• coalesced by SETA?: yes (often requires backfilling)
• available on platforms:
  • Linux: 64bit (OPT, PGO builds)
  • Windows: 7 32bit, 10 64bit (OPT, PGO builds)
  • OS X: 10.10 (OPT builds only)
• triaging specifics:

build_metrics

• short description: Monitor build times on multiple platforms, the size of the installers and other compiler-specific insights.
• frequency: little less than daily, around 5 alerts
• contact: :froydnj, :ted.mielczarek, :gps
• coalesced by SETA?: no (doesn't require backfilling)
• available on platforms:
  • Linux: 32/64bit (OPT, No-OPT builds)
  • Windows: 32/64bit (OPT, No-OPT, Mingw builds)
  • OS X: 10.10 (cross, no-cross builds)
  • Android: 4.0, 4.2, 5.0
• triaging specifics:
  • often easy to investigate
  • most alerts aren't noisy
  • when investigating, one should look for build config changes <ask :gps to provide more data>
  • build times often spike upwards for just a short time; they then lower to previous levels thanks to caching mechanisms set in place. We mark these as invalid alerts.

AWSY

• short description: link
• frequency: every 2-3 days, around half a dozen
• contact: :erahm
• coalesced by SETA?: yes
• available on platforms:
  • Linux: 64bit (OPT builds)
  • Windows: 32/64bit (OPT, PGO builds)
  • OS X: 10.10 (OPT builds)
  • Android: 4.2, 4.3 (OPT builds)
• triaging specifics:
  • retriggering/backfilling takes some time (>1h per test), so one must not abuse in collecting missing graph data