QA/Desktop Firefox/Triaged Bug List: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
m (updated whiteboard tags to current set of flags)
 
(13 intermediate revisions by 5 users not shown)
Line 1: Line 1:
= QA+ Triage Strategy for Bug Verifications =
= QA+ Triage Strategy for Bug Verifications =
In the Whiteboard field in the bug we'll use '''[qa+]''', '''[qa-]'''', '''[qa?]''' and '''[qa!]''' according to the following criteria:


* qa+: QA needs to verify the fix
We use this dashboard for triaging the current versions of Firefox:
** nightly: VERIFIED FIXED status
https://crash-analysis.mozilla.com/rkaiser/crash-report-tools/qa/
** aurora: verified-aurora keyword
 
** beta: verified-beta keyword
Mark bugs for verification with the qe-verify flag.
 
 
* qe-verify?: More information is needed for QA to decide on qe+/-
** if a bug is missing a testcase or steps to reproduce
** if a bug is hard to understand, too complex
** Canned Response:
*** qe-verify?: We need more information to verify this bug.
* qe-verify+: QA needs to verify the fix
** the test case to verify the fix should be clearly defined
** the test case to verify the fix should be clearly defined
** Canned Response:
** Canned Response:
*** qa+: tracking to verify fix (indicate branches needed for verification)
*** qe-verify+: tracking to verify fix (indicate branches needed for verification)
* qa-: QA can/will/need not verify the fix
* qe-verify-: QA can/will/need not verify the fix
** build config bugs
** build config bugs
** code-cleanup bugs
** code-cleanup bugs
Line 15: Line 22:
** bugs with the assertion keyword
** bugs with the assertion keyword
** Canned Response:
** Canned Response:
*** qa-: nothing for QA to verify
*** qe-verify-: nothing for QA to verify
* qa?: More information is needed for QA to decide on qa+/-
* qe-verify!: QA has done all it can to test this bug
** if a bug is missing a testcase or steps to reproduce
** if a bug is hard to understand, too complex
** Canned Response:
*** qa?:
* qa!: QA has done all it can to test this bug
** verified across all affected branches and platforms
** verified across all affected branches and platforms
** made a reasonable effort reaching out to domain experts or bug reporters who, in the absence of clear steps to reproduce or test cases, can verify the bug
** made a reasonable effort reaching out to domain experts or bug reporters who, in the absence of clear steps to reproduce or test cases, can verify the bug


We will be triaging bugs on a regular, weekly basis, from a list pulled from hg to include a week or two weeks worth of checkins.
(Out of date?)
* qa^: QA would like higher prioritization on this bug from dev:
** qa has seen this bug and triaged but the bug has not been dev prioritized
** QA has seen this bug and dev priority and qa priority does not match due to end user usability issues.
** This is mostly used for the mobile team, as dev prioritizes the bugs with dev priority using the priority field.


== Bug Query Mechanics ==
== Bug Query Mechanics ==
You can use the dashboard at https://crash-analysis.mozilla.com/rkaiser/crash-report-tools/qa/
Or...
Go to the pushlog page and query for the last 2 weeks worth of bugs depending on how recently you have had a triage session. Then apply the "collect buglinks" bookmarklet to get a list of bugs from the pushlog, and go through those.
Go to the pushlog page and query for the last 2 weeks worth of bugs depending on how recently you have had a triage session. Then apply the "collect buglinks" bookmarklet to get a list of bugs from the pushlog, and go through those.


Line 33: Line 43:
** Visit this link and right-click on the fourth bookmarklet item from the top. Add this to your Bookmarks Toolbar.
** Visit this link and right-click on the fourth bookmarklet item from the top. Add this to your Bookmarks Toolbar.


* [https://hg.mozilla.org/releases/mozilla-aurora/pushloghtml Aurora Pushlog Page]
* [https://hg.mozilla.org/releases/mozilla-aurora/pushloghtml Aurora Pushlog Page 1-week to Now]
** [https://hg.mozilla.org/releases/mozilla-aurora/pushloghtml?startdate=2+week+ago&enddate=now 2-weeks to Now]
** [https://hg.mozilla.org/releases/mozilla-aurora/pushloghtml?startdate=2+week+ago&enddate=now 3-weeks to Now]
** Visit this page and adjust the query for 1 week or more.
** Visit this page and adjust the query for 1 week or more.
** Click on the bookmarklet you added on your bookmarks toolbar. This should produce a list of bugs from all the checkins in the pushlog page.
** Click on the bookmarklet you added on your bookmarks toolbar. This should produce a list of bugs from all the checkins in the pushlog page.


Go through this list which should contain a list of recently Fixed bugs and apply the criteria above to triage them.
Go through this list which should contain a list of recently Fixed bugs and apply the criteria above to triage them.
== Triage Meetings ==
; Past Meetings
* [[QA/Desktop_Firefox/Triaged_Bug_List/Meetings/2011-10-26|October 26, 2011]]

Latest revision as of 18:09, 8 September 2014

QA+ Triage Strategy for Bug Verifications

We use this dashboard for triaging the current versions of Firefox: https://crash-analysis.mozilla.com/rkaiser/crash-report-tools/qa/

Mark bugs for verification with the qe-verify flag.


  • qe-verify?: More information is needed for QA to decide on qe+/-
    • if a bug is missing a testcase or steps to reproduce
    • if a bug is hard to understand, too complex
    • Canned Response:
      • qe-verify?: We need more information to verify this bug.
  • qe-verify+: QA needs to verify the fix
    • the test case to verify the fix should be clearly defined
    • Canned Response:
      • qe-verify+: tracking to verify fix (indicate branches needed for verification)
  • qe-verify-: QA can/will/need not verify the fix
    • build config bugs
    • code-cleanup bugs
    • unsupported platforms (ie. OS/2, Solaris, PPC Mac, etc)
    • bugs with the assertion keyword
    • Canned Response:
      • qe-verify-: nothing for QA to verify
  • qe-verify!: QA has done all it can to test this bug
    • verified across all affected branches and platforms
    • made a reasonable effort reaching out to domain experts or bug reporters who, in the absence of clear steps to reproduce or test cases, can verify the bug

(Out of date?)

  • qa^: QA would like higher prioritization on this bug from dev:
    • qa has seen this bug and triaged but the bug has not been dev prioritized
    • QA has seen this bug and dev priority and qa priority does not match due to end user usability issues.
    • This is mostly used for the mobile team, as dev prioritizes the bugs with dev priority using the priority field.

Bug Query Mechanics

You can use the dashboard at https://crash-analysis.mozilla.com/rkaiser/crash-report-tools/qa/

Or...

Go to the pushlog page and query for the last 2 weeks worth of bugs depending on how recently you have had a triage session. Then apply the "collect buglinks" bookmarklet to get a list of bugs from the pushlog, and go through those.

  • Jesse's Bookmarklets Page
    • Visit this link and right-click on the fourth bookmarklet item from the top. Add this to your Bookmarks Toolbar.

Go through this list which should contain a list of recently Fixed bugs and apply the criteria above to triage them.

Triage Meetings

Past Meetings
Retrieved from "https://wiki.allizom.org/index.php?title=QA/Desktop_Firefox/Triaged_Bug_List&oldid=1013128"