QA/Execution/Web Testing/roles/buildmaster
< QA | Execution | Web Testing
Build Master
Create a set of guidelines for investigating failures in our public Jenkins instance. This would be used by whoever is monitoring for failures, but would also be valuable for community who want to dive in. It should include how to identify failures, how to determine if they're already known, how to replicate them locally, how to determine if they're application bugs or test bugs, where to raise them, who to notify, and even how to fix them (if they're test failures) and submit pull requests. This could form part of a boot camp similar to other teams. -- from our 2015, Q2 goals brainstorm.
Open Questions
- In the interests of reducing complexity, do we need tiers? Are we anticipating that there will be so many failures that some need prioritizing over others?
- On the topic of sending daily emails - I think this would be extra work and considered noise by most recipients. It could be done via a whiteboard entry and Bugzilla whines.
- On the topic of checking builds once a day - There are several methods for doing this: view the web dashboard, subscribe to RSS feeds, read e-mail alerts, watch IRC notifications. We could consider others using Jenkins plugins.
Owner
- Current owner: stephend
- Next owner: rbillings
Role Definition
- build master role last 2 weeks
- edit jenkins desc with your name as buildmaster
- build master is point of contact for open issues/bugs
- role includes filing bugs/issues, sending out emails, investigating issues
- does NOT include escalation paths, prioritizing fixes, following up with other teams
- send daily email with list of generally prioritized github issues to be fixed, blocking bugs that were filed
- check builds at least once per day (there are several methods for doing this: view the web dashboard, subscribe to RSS feeds, read e-mail alerts, watch IRC notifications. We could consider others using Jenkins plugins)
- investigate failures
- If it is a locator issue, if you have a question, if you wonder if the test is still valid or important then file a GitHub issue
- File bugs on projects based on info below, then contact the noted team members
Support Tiers
Tier 1
- Marketplace
- AMO
- Mozilla.org
Tier 2
- SUMO
- Socorro
Tier 3
- BIDPOM
- Affiliates
- Mozillians
- Moztrap
- One and Done
Projects
amo
- high priority
- fails: 1st krupa / 2nd mathieu agopian FRANCE / 3rd wil clouser
- file a bug
bidpom
- towards low priority
- fail: john morrison [jrgm] if infrastructure related (time outs, buttons not loading, etc.); bob or davehunt are the ones to fix
- IRC: mozwebqa
- known bug file bug and also need info him; esp if you know who checked in the change who made it fail
Marketplace
- high priority
- fails: krupa 1st / 2nd david durst EST/ wil clouser 3rd
- IRC: #marketplace
- check
- file a bug - if we know there's been a regression, if you can reproduce manually and you can correlate with what you see in automation, or if it's infrastructure related
mozilla.org
- medium-high priority
- fails: pmac/jgmize
- IRC: www
- check
- file bug
Moztrap
- lower priority
- fails: peterbe/camd
- IRC: moztrap
- file a bug
mozwebqa dashboard
- low priority
- fail: bob
- IRC: mozwebqa
- https://github.com/mozilla/mozwebqa-dashboard/issues
One and Done
- low priority
- bob
- https://bugzilla.mozilla.org/enter_bug.cgi?component=One%20and%20Done&product=Mozilla%20QA
Socorro
- low priority
- tests will be pared down to match less upcoming features
- fails:
- contact: mbrandt and rhelmer in #breakpad
- https://bugzilla.mozilla.org/enter_bug.cgi?product=Socorro
Sumo
- low priority
- fails: Mythmon, rdalal, r1cky [who is leaving SUMO]
- contact: IRC sumodev
- known bug file bug and also need info him; esp if you know who checked in the change who made it fail
QMO
- low priority
- tests not in jenkins
- could make a public job for contributors to workon or fix
- https://bugzilla.mozilla.org/enter_bug.cgi?product=quality.mozilla.org