canmove, Confirmed users
1,126
edits
Sheriffing/Job Visibility Policy: Difference between revisions
m
→8) Must avoid patterns known to cause non deterministic failures: Tweak wording
(→8) Must avoid patterns known to cause non deterministic failures: Mention MOZ_DISABLE_NONLOCAL_CONNECTIONS and automation prefs) |
m (→8) Must avoid patterns known to cause non deterministic failures: Tweak wording) |
||
| Line 64: | Line 64: | ||
** Since these will cause failures when the external site is unavailable, as well as impacting end to end times & adding noise to performance tests. | ** Since these will cause failures when the external site is unavailable, as well as impacting end to end times & adding noise to performance tests. | ||
** eg: Emulator/driver binaries direct from a vendor's site, package downloads from PyPi or page assets for unit/performance tests. | ** eg: Emulator/driver binaries direct from a vendor's site, package downloads from PyPi or page assets for unit/performance tests. | ||
** | ** Ensure MOZ_DISABLE_NONLOCAL_CONNECTIONS is defined in the automation environment (see bug 995417) & use a list of automation prefs for switching off undesirable behaviour (eg automatic updates, telemetry pings; see bug 1023483 for where these are set). | ||
* Must not contain time bombs, e.g. tests that will fail after a certain date or when run at certain times. | * Must not contain time bombs, e.g. tests that will fail after a certain date or when run at certain times. | ||
* See the [https://developer.mozilla.org/en-US/docs/Mozilla/QA/Avoiding_intermittent_oranges best practices for avoiding intermittent failures (oranges)]. | * See the [https://developer.mozilla.org/en-US/docs/Mozilla/QA/Avoiding_intermittent_oranges best practices for avoiding intermittent failures (oranges)]. | ||