Confirmed users
4,072
edits
Litmus:How to Triage Failed and Unclear test cases: Difference between revisions
Litmus:How to Triage Failed and Unclear test cases (view source)
Revision as of 16:59, 5 November 2008
, 5 November 2008→Review Failed results daily
mNo edit summary |
|||
| Line 14: | Line 14: | ||
Once Failed results are found from the above formats, review each result by clicking the link to the result report. For each result reported: | Once Failed results are found from the above formats, review each result by clicking the link to the result report. For each result reported: | ||
* Confirm or deny the validity of the result. Results are automatically marked as Valid. It is up to the reviewer to determine which results are actually valid or invalid as Failed. Results filed with an associated bug are likely to remain valid. Results filed with good comments should be closely reviewed, even tested to confirm validity. Results filed with no comments or associated bugs can be subjectively reviewed as to severity and known frequency of report. | * Confirm or deny the validity of the result. Results are automatically marked as Valid. It is up to the reviewer to determine which results are actually valid or invalid as Failed. Results filed with an associated bug are likely to remain valid. Results filed with good comments should be closely reviewed, even tested to confirm validity. Results filed with no comments or associated bugs can be subjectively reviewed as to severity and known frequency of report. | ||
** If a result is valid as ''Failed'' | ** If a result is valid as '''Failed''': | ||
*** Make sure there is a bug on file for it in bugzilla.mozilla.org (check for one already filed, if not, create a new bug). | |||
*** Then leave the Valid?: box checked in the result report. | |||
*** Add a bug reference to the report under referenced bugs. Enter the bug number, then click '''Add Bug References''' | |||
** If the result is invalid, un-check the box in the Valid?: field of the result. | ** If the result is invalid, un-check the box in the Valid?: field of the result. | ||
* Vet the result. Vetting is a way to mark results as having been reviewed and action taken. Once you are done with result validation and action taken, click the '''Vet Result''' button in the result report. | * Vet the result. Vetting is a way to mark results as having been reviewed and action taken. Once you are done with result validation and action taken, click the '''Vet Result''' button in the result report. | ||
* Add a comment as to the action taken, then click '''Submit Comment''' | * Add a comment as to the action taken, then click '''Submit Comment''' | ||
* If the day of triage shows just a few valid reports, the lead can usually take on bug filing. However, if there are a lot of valid failed results, the Litmus lead should forward the results to the appropriate subgroup owner for them to take action. In which case, it will be their responsibility to file the bug then Vet the result. | |||
= Review Unclear results at least once a week = | = Review Unclear results at least once a week = | ||