Sheriffing/How To/Recommended Try Practices

The TryServer is a critical piece of our infrastructure for ensuring that patches are suitable for landing prior to being pushed. However, proper usage is important to avoid pushing under- or over-tested patches. This page is intended to provide some recommended practices for finding the right balance. The TryChooser Syntax Builder offers control over which builds and tests are run.

Note that the guidelines below are not a substitute for using your best judgement. The guidelines below are intended to be general recommendations but not absolute musts.

In many cases, a "try: -b do -p all -u all -t none" is overkill and should be avoided. A full Try run consumes over 500hrs of machine time to generate a full set of builds and run the full suite of tests across them. This can have a real impact on wait times across resource-constrained platforms, including production runs.
If in doubt, run both debug and opt builds.
If the patch only affects one platform, only that platform should have builds and tests.
- For example, a B2G-only Try run should look something like "try: -b do -p emulator,emulator-jb,emulator-kk,linux32_gecko,linux64_gecko,macosx64_gecko,win32_gecko -u all -t none"
Be aware of which test suites are likely to be affected by your patch and avoid running others that won't.

For example, a devtools-only change is unlikely to break any tests outside of mochitest-dt and xpcshell.

If the code is cross-platform, a T-style run (build on all platforms, test on one) will find most problems while making best use of resources.

T runs can be created by using the "Restrict tests to platform(s)" checkbox on TryChooser.
Linux32 is the best choice for the test platform as it is the least resource-constrained.

"try: -b do -p all -u all[Ubuntu,-x64] -t none" will run builds on all platforms and tests only on Linux32.

Multiple platforms can be specified as-needed (i.e. Linux + Android or any other combo as needed).

If you no longer need builds/tests from a run, cancel the remaining jobs.

Use the stop button on Treeherder's job details panel.
Unneeded jobs are a frequent source of frustration-inducing backlog.

Any existing jobs on a push can be retriggered without a new push.

Use the retrigger button on Treeherder's job details panel.
You can trigger as many additional runs as needed (i.e. trying to track down an intermittent failure).
Retriggering a build will cause all tests to be run again after the build completes.

High scores!