User:Lukasblakk/TryServerSyntax

Syntax for Custom Try Requests

The goal of this syntax is to allow user to select particular build/unittest/talos combinations in their hg commit message, a config.info file in the $topsrcdir or from a web interface.

Syntax

try: --build [d,o,do,all] --p [(desktop_platforms,...),none] --m [(mobile_platforms,...),none] --u [(test_suites,...),all] --t [(talos_suites,...),none]

This covers:

• Build Types both(do) || opt(o) || debug(d) || m-c all override (all)
• Platforms none || --p {linux,linux64,osx,osx64,win32,win64}
• Mobile none || --m {mobile-maemo5-qt, android, etc}
• Unittest Suites all || --u {crashtest,reftest,mochitest{1,2,3,4,5},xpcshell,etc}
• Talos Suites all || --t {tp4,tscroll,tspider,etc}

Usage Examples

In an hg commit message (currently active)

 # to run what mozilla-central's automation does ie: all platforms, all tests/talos
 hg commit -m "bug 432432 testing something try: --build all" 

 # both opt/debug builds, all available {desktop,mobile} platforms, no unittests/talos
 hg commit -m "bug 432432 testing something try: --build do" 

 # linux debug build, all available mobile platforms, all unittests, no talos
 hg commit -m "bug 432432 testing something try: --build d --p linux --u all"

 # mobile-only builds
 hg commit -m "bug 432432 testing something try: --build o --p none"

 # all builds, all available desktop platforms, no mobile, all unittests & talos
 hg commit -m "bug 432432 testing something try: --build do --m none --u all --t all"

 # opt builds on all available {desktop,mobile} platforms, no unittests, talos tp4 & tscroll 
 hg commit -m "bug 432432 testing something try: --build o --t tp4,tscroll"

In a try.info file (not yet active)

try.info file pushed to $topsrc_dir would look like:

 try: --build do --m none --u all --t all

 OR

 try: --build all

 OR

 try: --build d --m none --u reftest

Scheduler/Builder Parsing

• splits on 'try:'
• then splits on ' ' to create a list to pass to argparse which runs parse_known_args (thus keeping anything that is not recognized from causing errors)
• we also have allowed for manual sendchanges to send in comments with "try: ..." in order to make it possible to (re)run a particular test/talos/platform and not all things specified in a commit message

Notes on Usage

Default Set

If no try hg commit comment (ie: beginning with 'try:') is present then the default set is a "Does it compile?" run comprised of:

All try desktop platforms' debug builds, all mobile builds, no tests, no talos

Optional arguments

If the following exist they will be handled as follows:

build (default is debug)

Options are o (opt) || d (debug) || do (gets you both) || all (override for m-c style run which will do all desktop/mobile/test/talos too)
We use this to build the desktop and test builder names

desktop_platforms (if not present default is all) should be either comma-separated names with no spaces or 'none':

--p linux,macosx   ||  --p none 

mobile_platforms (if not present default is all) should be either comma-separated names with no spaces or 'none':

--m maemo5-gtk,android-r7 || --m none

unittests (if not present default is none) should be either comma-separated names with no spaces or 'all':

--u crashtest,jsreftest || --u all 

talos (if not present default is none) should be either comma-separated names with no spaces or 'none':

--t tp4,tscroll || --t all

Tutorial

• You've got your patch ready to try
• Set the commit comment to what you would like to run on try. In this case, let's say you want a linux opt build, no mobile, and only get talos tp4 results

hg commit -m "Patch to fix a pixel placement try: --build o --p linux --m none --t tp4"

• Push to try
• What happens? 'Linux tryserver build' builder is created and runs your opt linux build. If the build completes successfully then the talos tests are triggered and once the test-master gets the notice of the change it will read the hg commit comments and only kick off the one builder for tp4 'Rev3 Fedora 12 tryserver talos tp4'

What if I don't put try: in my commit comments?

If you do not specify anything in your comments you will get the default set of builds for try.

What if make a syntax error in my commit comments?

If you do not specify anything in your comments you will get the default set of builds for try.

How do I know what test/talos suites I can list?

Note: we are using a flat file for right now, LINK TO VALID_BUILDERS but bug 589847 has been filed and in the next iteration we will have a way to generate the lists of valid builder info dynamically from config.py which will keep it in sync with try's configs

What do I do if I need something run?

If you missed something in your commit comments you can ping whoever is on buildduty [link] and give them [details] to have the build/test/talos you missed on your changeset triggered. (or re-triggered)