304
edits
Regression Testing: Difference between revisions
Formatting
(Related products) |
(Formatting) |
||
| Line 1: | Line 1: | ||
This is dummy for article where I think to unite all information about regression testing of Mozilla's products. | This is dummy for article where I think to unite all information about regression testing of Mozilla's products. | ||
= Related Mozilla's Products = | == Related Mozilla's Products == | ||
* [[Thunderbird]] | * [[Thunderbird]] | ||
= Regression = | == Regression == | ||
When you notice that suddenly something breaks with a new build of Thunderbird, and you go back to an older Thunderbird build, and it works again, then you found a "regression". These are bugs that are introduced by a very recent change and what developers are generally very interested in fixing (or at least they should be). | When you notice that suddenly something breaks with a new build of Thunderbird, and you go back to an older Thunderbird build, and it works again, then you found a "regression". These are bugs that are introduced by a very recent change and what developers are generally very interested in fixing (or at least they should be). | ||
| Line 10: | Line 10: | ||
However, to do anything, we need a "regression range", and we need your help for that. The idea is to find a 1-day range of changes that broke things for you, which limits the amount of changes to look at. For that, you need to look at the nightlies and test a number of them. | However, to do anything, we need a "regression range", and we need your help for that. The idea is to find a 1-day range of changes that broke things for you, which limits the amount of changes to look at. For that, you need to look at the nightlies and test a number of them. | ||
= How to find a regression range using nightly builds = | == How to find a regression range using nightly builds == | ||
# Find the last nightly that works and first nightly that breaks. Do a so-called "binary search", e.g. try March 1 (works) and March 30 (fails), then the middle, i.e. March 15 (works), then again the middle, i.e. March 22 (fails), March 18 (works), March 20 (fails), March 19 (works). Now you know that it broke between March 19 and March 20. | # Find the last nightly that works and first nightly that breaks. Do a so-called "binary search", e.g. try March 1 (works) and March 30 (fails), then the middle, i.e. March 15 (works), then again the middle, i.e. March 22 (fails), March 18 (works), March 20 (fails), March 19 (works). Now you know that it broke between March 19 and March 20. | ||