TestEngineering/Performance/RunningTests: Difference between revisions
Alexandrui (talk | contribs) (→On try) |
(Improve prerequistives) |
||
| Line 1: | Line 1: | ||
{{DISPLAYTITLE:Running Performance Tests}} | |||
= | = Prerequisites = | ||
To be able to run one or more performance tests you first need to know which tests you would like to run. If you're responding to a regression bug then the test names will be listed in the bug report. You can also discover the performance tests and frameworks available from our [[TestEngineering/Performance#Projects|projects list]]. You can also see our new [https://firefox-source-docs.mozilla.org/testing/perfdocs/index.html performance test documentation], which will eventually replace our wiki pages. | |||
To run the tests you will need to have the [https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Source_Code source code] available locally, and satisfy the requirements for running [https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/mach mach] commands. If you want to run tests against the same infrastructure as our continuous integration then you will need to follow the documentation at [[ReleaseEngineering/TryServer]]. | |||
== Running tests == | == Running tests == | ||
Revision as of 10:59, 5 June 2020
Prerequisites
To be able to run one or more performance tests you first need to know which tests you would like to run. If you're responding to a regression bug then the test names will be listed in the bug report. You can also discover the performance tests and frameworks available from our projects list. You can also see our new performance test documentation, which will eventually replace our wiki pages.
To run the tests you will need to have the source code available locally, and satisfy the requirements for running mach commands. If you want to run tests against the same infrastructure as our continuous integration then you will need to follow the documentation at ReleaseEngineering/TryServer.
Running tests
Where?
As long as you have mach installed and configured, you can run a performance test locally or on try. But, first command you need to run to get familiar with the complexity of mach is:
$ ./mach help
On try
here’s the option on running a performance test on try (treeherder). This option will run the test on an integration environment from mozilla, meaning that it won’t use any of your workstation resources to get the results.
The command is:
$ ./mach try chooser
to get a nice UI to choose the tests to run, platform, build type
or
$ ./mach try fuzzy
to get a CLI on your console to choose the tests you want to run.
If you know exactly how to spell some words in the test signature, like platform, test name or build type, you might want to consider using the simple quote ‘ to filter up the results to your preference.
If you know exactly how the signature starts you can use a ^ in front of your search string and the results will narrow down to just that signature
If you want to skip the chooser or the fuzzy and to push the test to try directly from the command line, you can use the option -q (from query). The option --no-push will allow you to check the jobs you’re pushing by printing them to the command line without actually pushing them to try. The option -m is pretty handy when you push several times the same test (and believe me, you will!).
$ ./mach try fuzzy --full -q="'linux64 'raptor-tp6-1-firefox" -m="base for [bug/commit]" --no-push
And narrowing the results down
$ ./mach try fuzzy --full -q="'linux64-shippable/opt-raptor-tp6-1-firefox" -m="base for [bug/commit]" --no-push
Or just specifying how the test signature starts:
Once you found your query, you just need to remove the --no-push to push to try.
Those are just some examples of what performance sheriffs do when pushing tests to try. You can do anytime the command below to get the full help of the try command.
$ ./mach try --help