Confirmed users
4,072
edits
Litmus:How to Triage Failed and Unclear test cases: Difference between revisions
Litmus:How to Triage Failed and Unclear test cases (view source)
Revision as of 16:37, 5 November 2008
, 5 November 2008no edit summary
m (→Purpose) |
mNo edit summary |
||
| Line 14: | Line 14: | ||
Once Failed results are found from the above formats, review each result by clicking the link to the result report. For each result reported: | Once Failed results are found from the above formats, review each result by clicking the link to the result report. For each result reported: | ||
* Confirm or deny the validity of the result. Results are automatically marked as Valid. It is up to the reviewer to determine which results are actually valid or invalid as Failed. | * Confirm or deny the validity of the result. Results are automatically marked as Valid. It is up to the reviewer to determine which results are actually valid or invalid as Failed. | ||
** Results filed with an associated bug are likely to remain valid. Results filed with good comments should be closely reviewed, even tested to confirm validity. Results filed with no comments or associated bugs can be subjectively reviewed as to severity and | ** Results filed with an associated bug are likely to remain valid. Results filed with good comments should be closely reviewed, even tested to confirm validity. Results filed with no comments or associated bugs can be subjectively reviewed as to severity and known frequency of report. | ||
*** If a result is valid as ''Failed'', make sure there is a bug on file for it in bugzilla.mozilla.org (check for one already filed, if not, create a new bug). Then leave the Valid?: box checked in the result report. | *** If a result is valid as ''Failed'', make sure there is a bug on file for it in bugzilla.mozilla.org (check for one already filed, if not, create a new bug). Then leave the Valid?: box checked in the result report. | ||
*** If the result is invalid, un-check the box in the Valid?: field of the result. | *** If the result is invalid, un-check the box in the Valid?: field of the result. | ||
| Line 24: | Line 24: | ||
Once Unclear results are found from the above formats, review each result by clicking the link to the result report. For each result reported: | Once Unclear results are found from the above formats, review each result by clicking the link to the result report. For each result reported: | ||
* Confirm or deny the validity of the result. Results are automatically marked as Valid. It is up to the reviewer to determine which results are actually valid or invalid. If it is invalid, un-check the box in the Valid?: field of the result. | * Confirm or deny the validity of the result. Results are automatically marked as Valid. It is up to the reviewer to determine which results are actually valid or invalid. If it is invalid, un-check the box in the Valid?: field of the result. | ||
** Results filed with good comments should be closely reviewed, even tested to confirm validity. Results filed with no comments can be subjectively reviewed as to severity and | ** Results filed with good comments should be closely reviewed, even tested to confirm validity. Results filed with no comments can be subjectively reviewed as to severity and known frequency of report. | ||
*** If a result is valid as ''Unclear'', the action to be taken is to edit the test case associated to the report and make it clear. | *** If a result is valid as ''Unclear'', the action to be taken is to edit the test case associated to the report and make it clear. | ||
*** If the result is invalid, un-check the box in the Valid?: field of the result. | *** If the result is invalid, un-check the box in the Valid?: field of the result. | ||