MozillaWiki

QA/Execution/Web Testing/roles/buildmaster: Difference between revisions

Page Discussion

QA/Execution/Web Testing/roles/buildmaster (view source)

Revision as of 17:25, 5 January 2016

419 bytes removed ,  5 January 2016
no edit summary
No edit summary
Line 1: Line 1:
=Introduction=
=Introduction=
Create a set of guidelines for investigating failures in our public Jenkins instance. This would be used by whoever is monitoring for failures, but would also be valuable for community who want to dive in. It should include how to identify failures, how to determine if they're already known, how to replicate them locally, how to determine if they're application bugs or test bugs, where to raise them, who to notify, and even how to fix them (if they're test failures) and submit pull requests. This could form part of a boot camp similar to other teams. -- from our 2015, [https://etherpad.mozilla.org/webqa-goals-2015-q2 Q2 goals brainstorm].
The buildmaster role is assigned to a team member in rotation, each lasting two weeks. The role is intended to identify failures early, and to take action towards resolving the failures. This action will vary depending on the skills and knowledge of the current buildmaster, or the level of commitment the team has to the affected project.


=Open Questions=
=Monitoring Failures=
* In the interests of reducing complexity, do we need tiers? Are we anticipating that there will be so many failures that some need prioritizing over others?
Failures can be monitored in the following ways:
* On the topic of sending daily emails - I think this would be extra work and considered noise by most recipients. It could be done via a whiteboard entry and Bugzilla whines.
* [https://webqa-ci.mozilla.com/ Jenkins dashboard]
* On the topic of checking builds once a day - There are several methods for doing this: view the web dashboard, subscribe to RSS feeds, read e-mail alerts, watch IRC notifications. We could consider others using Jenkins plugins.
* [https://webqa-ci.mozilla.com/rssFailed RSS feed]
* [[/QA/Execution/Web_Testing/Automation/Build_Notifications|Mailing list]]
* [http://widget.mibbit.com/?settings=0a187437b06ca9bcb0cff2a717e985c5&server=irc.mozilla.org&channel=mozwebqa&noServerTab=false&autoConnect=true IRC]


=Rotation=
=Rotation=
{{:QA/Execution/Web Testing/roles/buildmaster/Rotation}}
{{:QA/Execution/Web Testing/roles/buildmaster/Rotation}}


=Definition=
==
* buildmaster role last 2 weeks
* A test is failing due to a locator issue
* edit jenkins desc with your name as buildmaster
* If it is a locator issue, if you have a question, if you wonder if the test is still valid or important then file a GitHub issue
* check builds at least once per day (there are several methods for doing this: view the web dashboard, subscribe to RSS feeds, read e-mail alerts, watch IRC notifications. We could consider others using Jenkins plugins)
** investigate failures
** If it is a locator issue, if you have a question, if you wonder if the test is still valid or important then file a GitHub issue
** File bugs on projects based on info below, then contact the noted team members
** File bugs on projects based on info below, then contact the noted team members
* File Git Issues for test failures that require a test update.
* File Git Issues for test failures that require a test update.
Line 28: Line 27:
For the latest known issues check [https://public.etherpad-mozilla.org/p/webqa-buildmaster this etherpad]. If you are the current buildmaster please try to keep this pad updated. It helps when it comes to hand the role onto the next buildmaster, and can avoid duplicating effort investigating failures.
For the latest known issues check [https://public.etherpad-mozilla.org/p/webqa-buildmaster this etherpad]. If you are the current buildmaster please try to keep this pad updated. It helps when it comes to hand the role onto the next buildmaster, and can avoid duplicating effort investigating failures.


=Support Tiers=
=New Failures=
 
When a new failure is found, details should be added to [https://public.etherpad-mozilla.org/p/webqa-buildmaster this etherpad] and an issue should be raised in the appropriate place (see projects section for details). This is the minimum expected of the buildmaster, however if you have the time and skills (or time to learn the skills), it's a good idea to try to replicate the failure locally. If the failure is clearly something that should be fixed in the test suite, then it's also appreciated if you can submit a pull request with a fix.
===Tier 1===
* Marketplace
* AMO
* Mozilla.org
 
===Tier 2===
* SUMO
* Socorro
 
===Tier 3===
* BIDPOM
* Moztrap
* One and Done
 
===Tier 4 (Unsupported)===
* Affiliates
* Mozillians


=Projects=
=Projects=
Davehunt
Confirmed users
2,177

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/1111422"