View source for User:Ashughes/Proposals/Good-First-Bug-Verifications

= Proposal for Bug Fix Verification Community Mentorship =

== Goal ==
The purpose of this proposal is to provide a more accessible way for new community members to learn about and contribute to the process of verifying bug fixes.

== Current Process ==
* QA Release Driver triages a list of status-firefoxN:fixed bugs
* Any bug deemed important enough to test before releasing the next Firefox version is flagged for testing using the ''verifyme'' keyword
* Similarly, the Driver will also triage the hg pushlog for a given milestone to flag important bugs with the ''verifyme'' keyword
* The testing team is then advised to verify all the ''verifyme'' flagged bugs before QA signs off a release

== The Problem ==
Poor visibility to outsiders of:
* how to contribute to the qualification of a release
* how to test and verify a bug fix
* what it means to mark a bug verified
* who to talk to and ask questions

== Proposal ==
Until we can finalize an official proposal I recommend the following ''intermediate'' proposal to be implemented as of Firefox 27 Nightly and iterated upon as it's put into practice.

Responsibilities
* '''QA Driver'''
** triage fixes as the land on Nightly and/or uplifted to Aurora/Beta as per details below
** ensure whiteboard tags, keywords, status flags, and resolution appropriately set
** educate testers and developers to ease maintenance over time
** find mentors for unassigned '''[good first verify]''' bugs

* '''Mentors''' (ie. teaching newcomers how to verify fixes)
** add "QA Mentor" to your name in Bugzilla
** set yourself as QA Contact on any bug you wish to mentor someone on
** engage with a person ASAP when they reach out on IRC, mailing list, or in Bugzilla
** provide guidance, including step by step instructions if necessary, to verify a bug
** provide assistance when they get stuck but try not to do the work for them
** continue to provide mentorship and tasks (ie. more bugs) until they become self sufficient

* '''Experienced Testers''' 
** test high risk, high importance, and/or time sensitive fixes as identified by the verifyme keyword
** set status flags and resolution to VERIFIED FIXED once verified across all patched branches
** assign themselves as mentors to '''[good first verify]''' bugs by using the QA Contact field
** provide guidance, encouragement, and appreciation to people who identify themselves as wanting to help
** push people who help to verify more fixes

* '''Bug Reporter'''
** test fixes we were unable to reproduce or which require an uncommon configuration; as identified by the '''need-info?''' flag
** set status flags and resolution to VERIFIED FIXED once verified across all patched branches; QA Driver to assist as needed

* '''Inexperienced Testers'''
** introduce themselves in '''[good first verify]''' bugs they want to test by adding a comment to the bug
** if a mentor is unassigned the QA Driver should find someone
** if a mentor is assigned the mentor should provide the tester as much guidance as needed to verify the patch across all branches

Tags, Keywords, Flags, etc
* '''verifyme''' ''(keyword)'' - fixes deemed high risk, important, and/or time sensitive; to be verified by experienced testers
* '''[qa-]''' ''(whiteboard)'' - fixes deemed unnecessary or impossible to verify (ex. in-testsuite, new APIs, test failures, etc)
* '''need-info?''' ''(flag)'' - fixes where the reporter's assistance is required (ex. uncommon configuration); to be verified by the reporter
* '''[good first verify]''' ''(whiteboard)'' - fixes deemed low risk and straightforward to test, ideal for inexperienced testers to learn about and help with verifications

Evangelism and Education
* integrate into a ''Get Involved'' section to Firefox test plans
* reach out periodically to QMO and mailing lists to solicit for volunteers
* educate developers and testers of the use of [qa-], verifyme, and [good first verify]
* incorporate [good first verify] into bugday and testday events