canmove, Confirmed users
1,126
edits
Buildbot/Talos/Tests: Difference between revisions
Bug 1082602
(Bug 1082602) |
|||
| Line 3: | Line 3: | ||
== Where to get this information == | == Where to get this information == | ||
* Talos tests are defined in | * Talos tests are defined in https://hg.mozilla.org/build/talos/file/tip/talos/test.py | ||
* | * Treeherder abbreviations are defined in https://github.com/mozilla/treeherder-service/blob/master/treeherder/etl/buildbot.py#L796 | ||
* [http://graphs.mozilla.org/ Perf-o-matic] names are defined in | * [http://graphs.mozilla.org/ Perf-o-matic] names are defined in https://hg.mozilla.org/graphs/file/tip/sql/data.sql | ||
* Talos suites are configured for production in | * Talos suites are configured for production in https://hg.mozilla.org/build/buildbot-configs/file/tip/mozilla-tests/config.py; these names are mapped to Treeherder via regexes: https://github.com/mozilla/treeherder-service/blob/master/treeherder/etl/buildbot.py#L344 | ||
* Inline help is available by running ''talos --print-tests'' to get a list of all tests and their descriptions. To get help on a subset of tests, including their run-time parameters, use e.g. ''talos --print-tests -a ts:tsvg [options]'' | * Inline help is available by running ''talos --print-tests'' to get a list of all tests and their descriptions. To get help on a subset of tests, including their run-time parameters, use e.g. ''talos --print-tests -a ts:tsvg [options]'' | ||
A table detailing information flow from buildbot to talos to | A table detailing information flow from buildbot to talos to Treeherder and graphserver is available at http://k0s.org:8080/ . This is generated with the [http://k0s.org/mozilla/hg/talosnames talosnames] script, as detailed in http://k0s.org/mozilla/blog/20120724135349 . See also {{bug|770460}}. | ||
== Talos Test Types == | == Talos Test Types == | ||
| Line 49: | Line 49: | ||
* -tpchrome tells Talos to run the browser with the normal browser UI active | * -tpchrome tells Talos to run the browser with the normal browser UI active | ||
* -tpnoisy means "generate lots of output" | * -tpnoisy means "generate lots of output" | ||
* -tpformat controls the format of the results, they default to the results we send to displays like [http://graphs.mozilla.org graphserver] and [ | * -tpformat controls the format of the results, they default to the results we send to displays like [http://graphs.mozilla.org graphserver] and [https://treeherder.mozilla.org Treeherder]. | ||
* -tpcycles controls the number of times we run the entire test. | * -tpcycles controls the number of times we run the entire test. | ||
| Line 496: | Line 496: | ||
* reporting: Summary of read/write counters for disk, network (lower is better) | * reporting: Summary of read/write counters for disk, network (lower is better) | ||
Talos will turn orange for 'x' jobs on windows 7 if your changeset accesses files which are not predefined in the [[http://hg.mozilla.org/build/talos/file/ae09eb897102/talos/xtalos/xperf_whitelist.json whitelist]] during startup; specifically, before the "[http://hg.mozilla.org/mozilla-central/file/0eebc33d8593/toolkit/components/startup/nsAppStartup.cpp#l631 sessionstore-windows-restored]" Firefox event. If your job turns orange, you will see a list of files in | Talos will turn orange for 'x' jobs on windows 7 if your changeset accesses files which are not predefined in the [[http://hg.mozilla.org/build/talos/file/ae09eb897102/talos/xtalos/xperf_whitelist.json whitelist]] during startup; specifically, before the "[http://hg.mozilla.org/mozilla-central/file/0eebc33d8593/toolkit/components/startup/nsAppStartup.cpp#l631 sessionstore-windows-restored]" Firefox event. If your job turns orange, you will see a list of files in Treeherder (or in the log file) which have been accessed unexpectedly (similar to this): | ||
* TEST-UNEXPECTED-FAIL : xperf: File '{profile}\secmod.db' was accessed and we were not expecting it. DiskReadCount: 6, DiskWriteCount: 0, DiskReadBytes: 16904, DiskWriteBytes: 0 | * TEST-UNEXPECTED-FAIL : xperf: File '{profile}\secmod.db' was accessed and we were not expecting it. DiskReadCount: 6, DiskWriteCount: 0, DiskReadBytes: 16904, DiskWriteBytes: 0 | ||
* TEST-UNEXPECTED-FAIL : xperf: File '{profile}\cert8.db' was accessed and we were not expecting it. DiskReadCount: 4, DiskWriteCount: 0, DiskReadBytes: 33288, DiskWriteBytes: 0 | * TEST-UNEXPECTED-FAIL : xperf: File '{profile}\cert8.db' was accessed and we were not expecting it. DiskReadCount: 4, DiskWriteCount: 0, DiskReadBytes: 33288, DiskWriteBytes: 0 | ||