ReleaseEngineering/Mozharness/How to run tests as a developer

The main thing you need is to checkout mozharness and call it like on tbpl, however, you will need to use these options:

• --no-read-buildbot-configs (as you're running it without buildbot)
• --installer-url (The Firefox binaries)
• --test-url (The tests zip)

hg clone http://hg.mozilla.org/build/mozharness scripts
# on Linux
python scripts/scripts/desktop_unittest.py --cfg unittests/linux_unittest.py --mochitest-suite plain1 --download-symbols ondemand --no-read-buildbot-config --installer-url http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-inbound-linux-debug/1391797503/firefox-30.0a1.en-US.linux-i686.tar.bz2 --test-url http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-inbound-linux-debug/1391797503/firefox-30.0a1.en-US.linux-i686.tests.zip
# on Mac
/tools/buildbot/bin/python scripts/scripts/desktop_unittest.py --cfg unittests/mac_unittest.py --reftest-suite reftest --download-symbols ondemand --no-read-buildbot-config --installer-url http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-inbound-win32/1370616247/firefox-24.0a1.en-US.win32.zip --test-url http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-inbound-win32/1370616247/firefox-24.0a1.en-US.win32.tests.zip
# on Windows
c:/mozilla-build/python27/python -u scripts/scripts/desktop_unittest.py --cfg unittests/win_unittest.py --reftest-suite reftest --download-symbols ondemand --no-read-buildbot-config --installer-url http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-inbound-win32/1370616247/firefox-24.0a1.en-US.win32.zip --test-url http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-inbound-win32/1370616247/firefox-24.0a1.en-US.win32.tests.zip

#emulator tests
python scripts/scripts/b2g_emulator_unittest.py --cfg b2g/emulator_automation_config.py --test-suite reftest --this-chunk 5 --total-chunks 10 --blob-upload-branch elm --download-symbols ondemand --no-read-buildbot-config --installer-url http://pvtbuilds.pvt.build.mozilla.org/pub/mozilla.org/b2g/tinderbox-builds/elm-generic/20140310074727/emulator.tar.gz --test-url http://pvtbuilds.pvt.build.mozilla.org/pub/mozilla.org/b2g/tinderbox-builds/elm-generic/20140310074727/b2g-30.0a1.en-US.android-arm.tests.zip

• If you use the above commands, you don't need a buildprops.json. Otherwise you need to create a buildprops.json (I think this goes in cwd); this is output in the log of the test you're trying to replicate.

Running Mozharness with a local version of Talos and Firefox on Linux

These instructions can be more general, but this should be a useful starting point for any similar configuration.

python scripts/talos_script.py --cfg configs/talos/linux_config_local.py

configs/talos/linux_config_local.py

PYTHON = '/home/v/work/talos2/bin/python'                                          
VENV_PATH = '/home/v/work/talos2'
FIREFOX = "/home/v/work/gecko-dev/obj-linux-optimized/dist/bin/firefox"
     
config = {  
    "talos_json": "talos.json",
    "use_talos_json": True,
    "suite": "custom",
    "python_webserver": True,
    "binary_path": FIREFOX,
    "log_name": "talos",
    "buildbot_json_path": "buildprops.json", #optional
    "virtualenv_path": VENV_PATH,
    "default_actions": [
        "clobber",
        "run-tests",
    ],      
}

talos.json

{                                                                                  
    "global": {
        "talos_repo": "/home/v/work/talos2",
        "talos_revision": "tip"
    },
    "suites": {
        "custom": {
            "tests": ["ts"]
        }
    },
    "mobile-suites": {
    }
}

Use this (and enable it in the config) if you want to simulate a try syntax commit message.

buildprops.json

{                                                                                  
  "sourcestamp": {
    "changes": [ {
          "comments": "try: simulated try parameters go here",
          "files": [ {"name":""} ]
        }
      ]
  }
}