Sheriffing/How To/Intermittent bugs: Difference between revisions
No edit summary |
|||
| Line 1: | Line 1: | ||
= How to file a Bug for a Test Failure aka Intermittent Bug = | |||
When you discover during your work on tree herder test failures who occurred only once (intermittent) we need a file a bug for this since the test might be need a change to make that test more stable. | When you discover during your work on tree herder test failures who occurred only once (intermittent) we need a file a bug for this since the test might be need a change to make that test more stable. | ||
| Line 12: | Line 12: | ||
Bascially 2 requirements need to be included in the bug that this bug can automatically picked up by treeheder: | Bascially 2 requirements need to be included in the bug that this bug can automatically picked up by treeheder: | ||
1. In the | 1. In the summary: '''Intermittent''' test_file test failure | ||
2. In the Keyword field choose the keyword: '''intermittent-failure''' | 2. In the Keyword field choose the keyword: '''intermittent-failure''' | ||
| Line 20: | Line 20: | ||
-> Open the Treeherder Log | -> Open the Treeherder Log | ||
1. Login into Bugzilla in a different tab | 1. Login into Bugzilla in a different tab | ||
2. Find the Product/Component where you need to file this bug (note: mxr and hg.mozilla.org can be very helpful if you are in doubt) | 2. Find the Product/Component where you need to file this bug (note: mxr and hg.mozilla.org can be very helpful if you are in doubt) | ||
3. In this example the Bug belong to Core:Document Navigation | 3. In this example the Bug belong to Core:Document Navigation | ||
4. In the Summary file the Bug as title: Intermittent navigation-timing/test_timing_xserver_redirect.html | expected OK | 4. Copy the failure text from the logwindow into the bug | ||
5. In the Summary: file the Bug as title: Intermittent navigation-timing/test_timing_xserver_redirect.html | expected OK | |||
6. In the keyword field choose intermittent-failure | |||
7. submit the bug | |||
in a perfect world the bug would like https://bugzilla.mozilla.org/show_bug.cgi?id=1172135 :) | |||
Treeherder syncs with bugzilla several times a day and so when this this is done from the next failure on your bug will be suggested for this failure! | |||
Revision as of 12:05, 16 June 2015
How to file a Bug for a Test Failure aka Intermittent Bug
When you discover during your work on tree herder test failures who occurred only once (intermittent) we need a file a bug for this since the test might be need a change to make that test more stable.
In a lot of cases the test failure happened before and someone else or maybe yourself filed a bug for that failure days or weeks ago. In such cases treeherder suggest the Bug Number and title under the failure like
Filing a bug
Filing a Bug so that its getting picked up by treeherder as in the screenshot before is very easy.
Bascially 2 requirements need to be included in the bug that this bug can automatically picked up by treeheder:
1. In the summary: Intermittent test_file test failure 2. In the Keyword field choose the keyword: intermittent-failure
Best Pratice
Lets imagine you have a test failure like TEST-UNEXPECTED-TIMEOUT | /navigation-timing/test_timing_xserver_redirect.html | expected OK in Treeherder and there is no bug filed so far for this failure.
-> Open the Treeherder Log
1. Login into Bugzilla in a different tab 2. Find the Product/Component where you need to file this bug (note: mxr and hg.mozilla.org can be very helpful if you are in doubt) 3. In this example the Bug belong to Core:Document Navigation 4. Copy the failure text from the logwindow into the bug 5. In the Summary: file the Bug as title: Intermittent navigation-timing/test_timing_xserver_redirect.html | expected OK 6. In the keyword field choose intermittent-failure 7. submit the bug
in a perfect world the bug would like https://bugzilla.mozilla.org/show_bug.cgi?id=1172135 :)
Treeherder syncs with bugzilla several times a day and so when this this is done from the next failure on your bug will be suggested for this failure!