Confirmed users
571
edits
Sheriffing/How To/Updating test expectations: Difference between revisions
Jump to navigation
Jump to search
Sheriffing/How To/Updating test expectations (view source)
Revision as of 10:21, 21 October 2022
, 21 October 2022→web-platform-tests: add detailed instructions about excluding .ini files from code getting reverted
No edit summary |
(→web-platform-tests: add detailed instructions about excluding .ini files from code getting reverted) |
||
| Line 48: | Line 48: | ||
# In the console, download the logs which a command similar to the following one:<br><code>cargo run mozilla-central e40a136dc8760f3fdd8a29fad47581d16d646c80 --out-dir ../logs/ --filter-jobs test-linux1804-32-shippable-qr/opt-web-platform-tests-e10s-10</code><br>Replace <code>mozilla-central</code> with the name of the tree and <code>e40a136dc8760f3fdd8a29fad47581d16d646c80</code> with the revision of the push from which the logs shall be downloaded.<br><code>--filter-jobs</code> expects the name of the task as parameter (found at the bottom left in Treeherder) - or a regular expression to match multiple task names. If omitted, the log for every wpt task for that push will be downloaded. Don't copy the job symbol mentioned at the end.<br>This depends on the Rust programming language which should be installed since <code>./mach bootstrap</code> had been run in <code>mozilla-unified</code> | # In the console, download the logs which a command similar to the following one:<br><code>cargo run mozilla-central e40a136dc8760f3fdd8a29fad47581d16d646c80 --out-dir ../logs/ --filter-jobs test-linux1804-32-shippable-qr/opt-web-platform-tests-e10s-10</code><br>Replace <code>mozilla-central</code> with the name of the tree and <code>e40a136dc8760f3fdd8a29fad47581d16d646c80</code> with the revision of the push from which the logs shall be downloaded.<br><code>--filter-jobs</code> expects the name of the task as parameter (found at the bottom left in Treeherder) - or a regular expression to match multiple task names. If omitted, the log for every wpt task for that push will be downloaded. Don't copy the job symbol mentioned at the end.<br>This depends on the Rust programming language which should be installed since <code>./mach bootstrap</code> had been run in <code>mozilla-unified</code> | ||
# Switch to <code>mozilla-unified</code><br><code>cd ../mozilla-unified/</code> | # Switch to <code>mozilla-unified</code><br><code>cd ../mozilla-unified/</code> | ||
# Update the files with the test expectations:<br><code> ./mach wpt-update ../logs/* --update-intermittent</code> | # Update the files with the test expectations:<br><code> ./mach wpt-update ../logs/* --update-intermittent</code>. This can take some time. | ||
# Check which files have been updated:<br><code>hg status</code><br>There are very likely more files modified (<code>M</code>) or added (<code>A</code>) than desired. | # Check which files have been updated:<br><code>hg status</code><br>There are very likely more files modified (<code>M</code>) or added (<code>A</code>) than desired. | ||
# Revert the unintentional changes:<br><code>hg revert -r . -X "path:testing/web-platform/meta/<continuation_of_path> | # Revert the unintentional changes:<br><code>hg revert -r . -X "path:testing/web-platform/meta/<continuation_of_path>"</code><br>Multiple paths can be kept in the current state by using the <code>-X</code> ("exclude") pattern multiple times.<br>'''<code><continuation_of_path></code> needs to be replaced with the path and file name of the failure message + ".ini".''' E.g. for <code>TEST-UNEXPECTED-PASS | example/test.html | should implement interface</code> use <code>path:testing/web-platform/meta/example/test.html.ini</code>. | ||
# Check <code>hg status</code> again. | # Check <code>hg status</code> again. | ||
# Save the changes with <code>hg commit -e</code> | # Save the changes with <code>hg commit -e</code> | ||