Confirmed users
1,759
edits
Auto-tools/Projects/Stockwell/disable-recommended: Difference between revisions
Auto-tools/Projects/Stockwell/disable-recommended (view source)
Revision as of 20:54, 22 February 2018
, 22 February 2018→This page is under construction!
| Line 1: | Line 1: | ||
= [stockwell disable-recommended] = | |||
When an intermittent-failure bug has recorded 150 or more failures in the last 21 days, the Orange Factor robot will change the bug's stockwell whiteboard tag to "[stockwell disable-recommended]". These bugs should be reviewed at least twice per week (current schedule is Monday and Thursday). | When an intermittent-failure bug has recorded 150 or more failures in the last 21 days, the Orange Factor robot will change the bug's stockwell whiteboard tag to "[stockwell disable-recommended]". These bugs should be reviewed at least twice per week (current schedule is Monday and Thursday). | ||
This page describes how to review [stockwell disable-recommended] bugs. | This page describes how to review [stockwell disable-recommended] bugs. | ||
== Finding bugs to disable == | |||
* Use bugzilla to find all bugs with whiteboard tag "[stockwell disable-recommended]". | * Use bugzilla to find all bugs with whiteboard tag "[stockwell disable-recommended]". | ||
* For each disable-recommended bug, determine if the bug can be addressed by disabling tests. | * For each disable-recommended bug, determine if the bug can be addressed by disabling tests. | ||
| Line 23: | Line 21: | ||
*** If there is a patch under review or comments indicate that a fix is coming soon, consider postponing action on this bug. | *** If there is a patch under review or comments indicate that a fix is coming soon, consider postponing action on this bug. | ||
== Finding the manifest == | |||
For each test to be disabled, find the associated test manifest: | For each test to be disabled, find the associated test manifest: | ||
* Search for the test name in searchfox or dxr, or, | * Search for the test name in searchfox or dxr, or, | ||
| Line 45: | Line 43: | ||
</pre> | </pre> | ||
== Determining which platforms are affected == | |||
On OrangeFactor, check the detail view for the bug for the last 30 days. Look down the list of bugs for affected Platforms and Build Types. Does this test need to be disabled on all platforms, or only on some? For opt or debug builds, or both? | On OrangeFactor, check the detail view for the bug for the last 30 days. Look down the list of bugs for affected Platforms and Build Types. Does this test need to be disabled on all platforms, or only on some? For opt or debug builds, or both? | ||
| Line 83: | Line 81: | ||
</pre> | </pre> | ||
The reason for this is we would be skipping on linux/windows/osx, but not linux32 or android. Since this test doesn't run on Android, we would only be skipping on linux32- and a test that only runs on 1 platform doesn't provide a lot of value unless it is platform specific. | The reason for this is we would be skipping on linux/windows/osx, but not linux32 or android. Since this test doesn't run on Android, we would only be skipping on linux32 - and a test that only runs on 1 platform doesn't provide a lot of value unless it is platform specific. | ||
== Updating a manifest to disable a test == | |||
There are 3 different types of manifest files, each with a different format: | There are 3 different types of manifest files, each with a different format: | ||
* "'''ini'''" manifests like mochitest.ini, chrome.ini, browser.ini, xpcshell.ini | * "'''ini'''" manifests like mochitest.ini, chrome.ini, browser.ini, xpcshell.ini | ||
| Line 158: | Line 156: | ||
+ if debug and (os == "win") and (version == "10.0.15063"): https://bugzilla.mozilla.org/show_bug.cgi?id=1435526 | + if debug and (os == "win") and (version == "10.0.15063"): https://bugzilla.mozilla.org/show_bug.cgi?id=1435526 | ||
== Making a patch == | |||
To actually disable a test, we need to update the manifest file in mozilla-central. Before doing that, create a mercurial patch containing the change, and upload the patch to bugzilla for review. | To actually disable a test, we need to update the manifest file in mozilla-central. Before doing that, create a mercurial patch containing the change, and upload the patch to bugzilla for review. | ||
| Line 187: | Line 185: | ||
NOTE: the comment for the commit needs to follow the general pattern above: "Bug xxx - <patch description>. r=yyy" | NOTE: the comment for the commit needs to follow the general pattern above: "Bug xxx - <patch description>. r=yyy" | ||
== Requesting review == | |||
In bugzilla, in the bug with [stockwell disable-recommended], use "Attach File" to attach your patch. Browse to find the patch in <your repo directory>/.hg/patches. Make sure you select "patch" for the Content Type. Then request review, "r?", and select a reviewer. In most cases, ask :gbrown or :jmaher for review. | In bugzilla, in the bug with [stockwell disable-recommended], use "Attach File" to attach your patch. Browse to find the patch in <your repo directory>/.hg/patches. Make sure you select "patch" for the Content Type. Then request review, "r?", and select a reviewer. In most cases, ask :gbrown or :jmaher for review. | ||
| Line 204: | Line 202: | ||
https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/How_to_Submit_a_Patch | https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/How_to_Submit_a_Patch | ||
== After the test is disabled == | |||
* "Watch" your patch landing to see if the patch was effective: | * "Watch" your patch landing to see if the patch was effective: | ||
** Are there any new failures on your push, or on the next few pushes, of the disabled test? | ** Are there any new failures on your push, or on the next few pushes, of the disabled test? | ||
* Or, check OrangeFactor (maybe the next day) to see if any new failures are reported. | * Or, check OrangeFactor (maybe the next day) to see if any new failures are reported. | ||
== Special cases / getting help == | == Special cases / getting help == | ||
There are lots of "special cases" -- different types of tests, platform variations, and who knows what! If something seems odd, if you need more information, we're here to help: Ping/email :gbrown / :jmaher. | There are lots of "special cases" -- different types of tests, platform variations, and who knows what! If something seems odd, if you need more information, we're here to help: Ping/email :gbrown / :jmaher. | ||