Confirmed users
1,759
edits
Auto-tools/Projects/Stockwell/backfill-retrigger: Difference between revisions
Auto-tools/Projects/Stockwell/backfill-retrigger (view source)
Revision as of 15:42, 29 March 2018
, 29 March 2018→exceptions and odd things
(→finding bugs to work on: - update url) |
|||
| (3 intermediate revisions by the same user not shown) | |||
| Line 3: | Line 3: | ||
As these are new bugs, there will be issues here that are infra or harness related. Use this as an opportunity to annotate [stockwell infra] if it is build, taskcluster, network, machine related. Otherwise, rules are similar to disable-recommended: If a test case is in the bugzilla summary, we should be able to retrigger and find the patch which caused this to become so frequent. | As these are new bugs, there will be issues here that are infra or harness related. Use this as an opportunity to annotate [stockwell infra] if it is build, taskcluster, network, machine related. Otherwise, rules are similar to disable-recommended: If a test case is in the bugzilla summary, we should be able to retrigger and find the patch which caused this to become so frequent. | ||
'''Skip test-verify bugs''': test-verify already repeats tests, and only runs tests which were modified on a push. There is no need to retrigger or backfill a test-verify failure. | |||
= choosing a config to test = | = choosing a config to test = | ||
| Line 69: | Line 71: | ||
When your initial tests finish, you might see a view like this: | When your initial tests finish, you might see a view like this: | ||
[[File:TH_repeat.jpg|500px]] | [[File:TH_repeat.jpg|500px]] | ||
Here you can see the 2-4 oranges per push. | Here you can see the 2-4 oranges per push. Check each failure to make sure the same test is failing. In the above case that is true and we need to go further back in history. | ||
After repeating the process a few times, the root cause will become visible: | After repeating the process a few times, the root cause will become visible: | ||
| Line 85: | Line 87: | ||
* root cause looks like a merge, repeat on the other integration branch | * root cause looks like a merge, repeat on the other integration branch | ||
* rarely but sometimes failures occur on mozilla-central landings, or as a result of code merging | * rarely but sometimes failures occur on mozilla-central landings, or as a result of code merging | ||
* sometimes it is obvious from check-in messages (or TV failures) that the failing test case was modified on a certain push: If the test was modified around the time it started failing, that's suspicious and can be used as a short-cut to find the regressing changeset. | |||