TestEngineering/Performance/Talos/Running: Difference between revisions

TestEngineering/Performance/Talos/Running (view source)

Revision as of 16:15, 12 August 2019

64 bytes added , 12 August 2019

no edit summary

Davehunt

Confirmed users

2,177

edits

@@ Line 3: / Line 3: @@
 {{todo|add [[../Profiling]]}}
 {{todo|add [[Mobile/Fennec/Android#talos|mobile]]}}
+= Desktop =
 == Locally ==
+{{warning|Firefox must have already been built successfully locally. See [https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/Simple_Firefox_build Building Firefox] for details.}}
-Prerequisite: Firefox has already been built successfully locally.
+[https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/mach mach] is used to run the [[../Tests|Talos tests]] locally. You can run suites of tests or individual tests on their own.
-[https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/mach Mach] is used to run the [[../Tests|Talos tests]] locally. You can run suites of tests or individual tests on their own.
 For example, to run the 'chromez' test suite:
-    ./mach talos-test --suite chromez
+  ./mach talos-test --suite chromez
 To run a single test (the 'tart' test):
-    ./mach talos-test --activeTests tart
+  ./mach talos-test --activeTests tart
-Certain talos suites/tests require external pagesets to be downloaded. If pagesets are required, they will be downloaded and extracted automatically when running talos via mach. To disable this automatic pageset downloading, add the 'no-download' argument:
+Certain Talos suites/tests require external pagesets to be downloaded. If pagesets are required, they will be downloaded and extracted automatically when running Talos via mach. To disable this automatic pageset downloading, add the 'no-download' argument:
-    ./mach talos-test --suite g1 --no-download
+  ./mach talos-test --suite g1 --no-download
-To see a list of available talos suites:
+To see a list of available Talos suites:
-    ./mach talos-test --print-suites
+  ./mach talos-test --print-suites
-To see a list of available talos tests:
+To see a list of available Talos tests:
-    ./mach talos-test --print-tests
+  ./mach talos-test --print-tests
 Further help running talos via mach:
-    ./mach talos-test --help
+  ./mach talos-test --help
 '''Tip:''' Clear the operating system cache between runs, in order to significantly improve the consistency of the results. For example, on linux:
-     find . | xargs touch
+  find . | xargs touch
-== Try Server ==
-To run talos tests on the try server, you just simply push to try as normal, specifying that you want to run Talos tests. It is highly recommended that you use [http://trychooser.pub.build.mozilla.org/ trychooser] to build your try server syntax (helpful hint for using trychooser: in the list of talos suites available you can hover over any one of the suites, and a list of tests included in that suite will be displayed).
-To run all of the talos tests as part of your try push, just add "-t all" to the end of your try syntax. To run a single talos suite as part of your try push, just add "-t <name of suite>" to your try syntax (i.e. "-t g1").
+== Try server ==
+To run Talos tests on the try server, you just simply push to try as normal, specifying that you want to run Talos tests. It is highly recommended that you use [http://trychooser.pub.build.mozilla.org/ trychooser] to build your try server syntax (helpful hint for using trychooser: in the list of Talos suites available you can hover over any one of the suites, and a list of tests included in that suite will be displayed).
-To turn on [[TestEngineering/Performance/Talos/Profiling|profiling]] as part of your talos tests on try, add "mozharness: --spsProfile" to the end of the try syntax (i.e. "-t g1 mozharness: --spsProfile"). There is an option to do this in trychooser.
+To run all of the Talos tests as part of your try push, just add "-t all" to the end of your try syntax. To run a single Talos suite as part of your try push, just add "-t <name of suite>" to your try syntax (i.e. "-t g1").
-When viewing your talos try server results on Treeherder, look for the group names 'T' and 'T-e10s'. Click on the green symbol corresponding to the talos suite name, and then click on the 'Performance' tab below to see the test results.
+To turn on [[../Profiling|profiling]] as part of your talos tests on try, add "mozharness: --spsProfile" to the end of the try syntax (i.e. "-t g1 mozharness: --spsProfile"). There is an option to do this in trychooser.
-If you're curious about the hardware we use, here is a [[../Misc#Hardware_Profile_of_machines_used_in_automation|description of the specs]].  You can also refer to the [[ReferencePlatforms#Build_Reference_Platforms|test reference platforms]] documentation.
+When viewing your Talos try server results on Treeherder, look for the group names 'T' and 'T-e10s'. Click on the green symbol corresponding to the Talos suite name, and then click on the 'Performance' tab below to see the test results.
-== Are my Talos Results OK? ==
+If you're curious about the hardware we use, here is a [[../Platforms|description of the specs]].  You can also refer to the [[ReferencePlatforms#Build_Reference_Platforms|test reference platforms]] documentation.
-The best way to answer this question is to push to try server and compare the performance of your push against a baseline, using Perfherder (it is recommended hitting retrigger a few times to get at least 5 data points for each test). After pushing to try with talos tests enabled, you will receive an email with a link to Perfherder where you can compare your results. Make sure you look at the different platforms.
+= Are my Talos results OK? =
+The best way to answer this question is to push to try server and compare the performance of your push against a baseline, using Perfherder (it is recommended hitting retrigger a few times to get at least 5 data points for each test). After pushing to try with Talos tests enabled, you will receive an email with a link to Perfherder where you can compare your results. Make sure you look at the different platforms.
 = Hacking on Talos Itself =