Confirmed users
356
edits
TestEngineering/Performance/Talos/RegressionBugsHandling: Difference between revisions
TestEngineering/Performance/Talos/RegressionBugsHandling (view source)
Revision as of 18:27, 9 February 2015
, 9 February 2015wording
m (wording tweak) |
m (wording) |
||
| Line 1: | Line 1: | ||
=Handling of Performance Regression Bugs= | =Handling of Performance Regression Bugs= | ||
In order to better understand and track our performance regressions, the performance team came up with the following system: | In order to better understand and track our Talos performance regressions, the performance team came up with the following system: | ||
===== When a performance regression is detected and the patch which caused it is identified ===== | ===== When a Talos performance regression is detected and the patch which caused it is identified ===== | ||
* Whoever landed the patch is responsible to drive a decision on whether or not the regression is acceptable. | * Whoever landed the patch is responsible to drive a decision on whether or not the regression is acceptable. | ||
* Such decision should preferably involve a module owner/peer of the regressed module. | * Such decision should preferably involve a module owner/peer of the regressed module. | ||
| Line 9: | Line 9: | ||
=====If the decision was to try and fix the regression, preferably a time frame and an owner would be provided as well.===== | =====If the decision was to try and fix the regression, preferably a time frame and an owner would be provided as well.===== | ||
=====It's OK to decide to | =====It's OK to decide to accept a Talos regression for valid reasons, including===== | ||
* There is other higher priority work, the patch's functionality is absolutely required, and no one else is available to investigate in the foreseeable future. | * There is other higher priority work, the patch's functionality is absolutely required, and no one else is available to investigate in the foreseeable future. | ||
* The magnitude of the regression is too small to warrant the effort required for an investigation. | * The magnitude of the regression is too small to warrant the effort required for an investigation. | ||
| Line 16: | Line 16: | ||
* etc. | * etc. | ||
=====Invalid reasons for closing a regression bug===== | =====Invalid reasons for closing a Talos regression bug===== | ||
* "I don't understand how my patch could regress this test" | * "I don't understand how my patch could regress this test" | ||
** If you think your patch is not to blame, we can re-trigger Talos test runs to confirm the before-and-after behavior. Talk to the person who filed the bug, they would be happy to help you figure out if the regression is real and if it is indeed your patch that is responsible. | ** If you think your patch is not to blame, we can re-trigger Talos test runs to confirm the before-and-after behavior. Talk to the person who filed the bug, they would be happy to help you figure out if the regression is real and if it is indeed your patch that is responsible. | ||