Sheriffing/How To/Recommended Try Practices: Difference between revisions

The [https://wiki.mozilla.org/ReleaseEngineering/TryServer TryServer] is a critical piece of our infrastructure for ensuring that patches are suitable for landing prior to being pushed. However, proper usage is important to avoid pushing under- or over-tested patches. This page is intended to provide some recommended practices for finding the right balance. The [http://trychooser.pub.build.mozilla.org/ TryChooser Syntax Builder] offers control over which builds and tests are run.

Note that the guidelines below are not a substitute for using your best judgement. The guidelines below are intended to be general recommendations but not absolute musts.

   <li>In many cases, a "try: -b do -p all -u all -t none" is overkill and <b>should be avoided</b>. A full Try run consumes <b><u>over 500hrs of machine time</u></b> to generate a full set of builds and run the full suite of tests across them. This can have a real impact on wait times across resource-constrained platforms, including production runs.</li>

   <li>If the patch only affects one platform, only that platform should have builds and tests.

  </li>

   <li>Be aware of which test suites are likely to be affected by your patch and avoid running others that won't.</li>

     <ul><li>For example, a devtools-only change is unlikely to break any tests outside of mochitest-dt and xpcshell.</li></ul>

   <li>If the code is cross-platform, a <i>T-style</i> run (build on all platforms, test on one) will find most problems while making best use of resources.

       <li>T runs can be created by using the "Restrict tests to platform(s)" checkbox on TryChooser.</li>

      <li>Linux32 is the best choice for the test platform as it is the least resource-constrained.</li>

      <li>Multiple platforms can be specified as-needed (i.e. Linux + Android or any other combo as needed).</li>

   <li>If you no longer need builds/tests from a run, cancel the remaining jobs.</li>