Sheriffing/How To/Intermittent bugs

From MozillaWiki
Jump to navigation Jump to search

How to file a Bug for a Test Failure aka Intermittent Bug

When you discover during your work on tree herder test failures who occurred only once (intermittent) we need a file a bug for this since the test might be need a change to make that test more stable.

In a lot of cases the test failure happened before and someone else or maybe yourself filed a bug for that failure days or weeks ago. In such cases treeherder suggest the Bug Number and title under the failure like

Treeherder suggestions Filing a bug

Filing a Bug so that its getting picked up by treeherder as in the screenshot before is very easy.

Bascially 2 requirements need to be included in the bug that this bug can automatically picked up by treeheder:

1. In the �summary: Intermittent test_file test failure 2. In the Keyword field choose the keyword: intermittent-failure

Best Pratice

Lets imagine you have a test failure like TEST-UNEXPECTED-TIMEOUT | /navigation-timing/test_timing_xserver_redirect.html | expected OK in Treeherder and there is no bug filed so far for this failure.

-> Open the Treeherder Log 1. Login into Bugzilla in a different tab 2. Find the Product/Component where you need to file this bug (note: mxr and hg.mozilla.org can be very helpful if you are in doubt) 3. In this example the Bug belong to Core:Document Navigation 4. In the Summary file the Bug as title: Intermittent navigation-timing/test_timing_xserver_redirect.html | expected OK 5.

Retrieved from "https://wiki.allizom.org/index.php?title=Sheriffing/How_To/Intermittent_bugs&oldid=1080276"