342
edits
TestEngineering/Performance/Sheriffing/Workflow: Difference between revisions
TestEngineering/Performance/Sheriffing/Workflow (view source)
Revision as of 12:03, 16 March 2022
, 16 March 2022→Handling improvements
Alexandrui (talk | contribs) |
Alexandrui (talk | contribs) |
||
| Line 203: | Line 203: | ||
Next you have to follow the comments of the bug so you make sure it’s closed, ideally before the next Firefox release. | Next you have to follow the comments of the bug so you make sure it’s closed, ideally before the next Firefox release. | ||
= Handling improvements = | == Handling improvements == | ||
Unlike for regressions, when you identified an improvement there's no need to open a bug, you just need to notify the bug assignee via a comment and add the 'perf-alert' keyword. | Unlike for regressions, when you identified an improvement there's no need to open a bug, you just need to notify the bug assignee via a comment and add the 'perf-alert' keyword. | ||
== Valid improvements == | === Valid improvements === | ||
This time you just need copy the summary, paste it as “Congrats” to the bug causing it and update the status of the summary: | This time you just need copy the summary, paste it as “Congrats” to the bug causing it and update the status of the summary: | ||
<br /> | <br /> | ||
| Line 221: | Line 221: | ||
'''Ticking the box next to the alert summary and resetting it will UNLINK the reassigned alerts and you don’t want to do that!''' | '''Ticking the box next to the alert summary and resetting it will UNLINK the reassigned alerts and you don’t want to do that!''' | ||
== Improvements treated as regression == | === Improvements treated as regression === | ||
Depending on the test, a high magnitude improvement (> 80%) should be treated more carefully. While a 100% improvement for a pageload test is impossible (the site never loads in an instant), over 80% is very rare and might imply that the test isn't loading what it should (an error page which is likely to contain much less code than the actual website). | Depending on the test, a high magnitude improvement (> 80%) should be treated more carefully. While a 100% improvement for a pageload test is impossible (the site never loads in an instant), over 80% is very rare and might imply that the test isn't loading what it should (an error page which is likely to contain much less code than the actual website). | ||