Syntax for Custom Try Requests

The goal of this syntax is to allow user to select particular build/unittest/talos combinations in their hg commit message, a config.info file in the $topsrcdir or from a web interface.

Syntax

try: --build [d,o,do,all] --p [(desktop_platforms,...),none] --m [(mobile_platforms,...),none] --u [(test_suites,...),all] --t [(talos_suites,...),none]

This covers:

Build Types both(do) || opt(o) || debug(d) || m-c all override (all)
Platforms none || --p {linux,linux64,osx,osx64,win32,win64}
Mobile none || --m {mobile-maemo5-qt, android, etc}
Unittest Suites all || --u {crashtest,reftest,mochitest{1,2,3,4,5},xpcshell,etc}
Talos Suites all || --t {tp4,tscroll,tspider,etc}

Usage Examples

In an hg commit message (currently active)

 # to run what mozilla-central's automation does ie: all platforms, all tests/talos
 hg commit -m "bug 432432 testing something try: --build all" 

 # both opt/debug builds, all available {desktop,mobile} platforms, no unittests/talos
 hg commit -m "bug 432432 testing something try: --build do" 

 # linux debug build, all available mobile platforms, all unittests, no talos
 hg commit -m "bug 432432 testing something try: --build d --p linux --u all"

 # mobile-only builds
 hg commit -m "bug 432432 testing something try: --build o --p none"

 # all builds, all available desktop platforms, no mobile, all unittests & talos
 hg commit -m "bug 432432 testing something try: --build do --m none --u all --t all"

 # opt builds on all available {desktop,mobile} platforms, no unittests, talos tp4 & tscroll 
 hg commit -m "bug 432432 testing something try: --build o --t tp4,tscroll"

In a try.info file (not yet active)

try.info file pushed to $topsrc_dir would look like:

 try: --build do --m none --u all --t all

 OR

 try: --build all

 OR

 try: --build d --m none --u reftest

Scheduler/Builder Parsing

splits on 'try:'
then splits on ' ' to create a list to pass to argparse which runs parse_known_args (thus keeping anything that is not recognized from causing errors)
we also have allowed for manual sendchanges to send in comments with "try: ..." in order to make it possible to (re)run a particular test/talos/platform and not all things specified in a commit message

Notes on Usage

Default Set

If you only specify try: or no commit comment is present (or info file) then the default set would be a "Does it compile?" set comprised of:

All try desktop platforms' debug builds, all mobile builds, no tests, no talos

Optional arguments

If the following exist they will be handled as follows:

build (default is debug)

Options are o (opt) || d (debug) || do (gets you both) || all (override for m-c style run which will do all desktop/mobile/test/talos too)
We use this to build the desktop and test builder names

desktop_platforms (if not present default is all) should be either comma-separated names with no spaces or 'none':

--p linux,macosx   ||  --p none

mobile_platforms (if not present default is all) should be either comma-separated names with no spaces or 'none':

--m maemo5-gtk,android-r7 || --m none

unittests (if not present default is none) should be either comma-separated names with no spaces or 'all':

--u crashtest,jsreftest || --u all

talos (if not present default is none) should be either comma-separated names with no spaces or 'none':

--t tp4,tscroll || --t all

How do I know what test/talos suites I can list?

Note: we are using a flat file for right now, LINK TO VALID_BUILDERS but bug 589847 has been filed and in the next iteration we will have a way to generate the lists of valid builder info dynamically from config.py which will keep it in sync with try's configs

What do I do if my commit comments don't get me what I want?

If you missed something in your commit comments you can ping whoever is on buildduty [link] and give them [details] to have the build/test/talos you've missed on your changeset triggered.

User:Lukasblakk/TryServerSyntax

Contents