canmove, Confirmed users
1,126
edits
Sheriffing/Job Visibility Policy: Difference between revisions
m
→7) Low intermittent failure rate: Grammar
m (→Requirements for being shown in the default TBPL view: Improve readability) |
m (→7) Low intermittent failure rate: Grammar) |
||
| Line 54: | Line 54: | ||
** Undermines devs confidence in the platform/test-suite - which as demonstrated by Firefox for Android, permanently affects their willingness to believe any future failures, even once the intermittent-failure rate is lowered. | ** Undermines devs confidence in the platform/test-suite - which as demonstrated by Firefox for Android, permanently affects their willingness to believe any future failures, even once the intermittent-failure rate is lowered. | ||
* A mozilla-central push results in ~400 jobs. The typical OrangeFactor across all trunk trees is normally (excluding the recent spike) 3-4, ie: a failure rate of ~1%. | * A mozilla-central push results in ~400 jobs. The typical OrangeFactor across all trunk trees is normally (excluding the recent spike) 3-4, ie: a failure rate of ~1%. | ||
* Therefore as a rough guide new platform/testsuite must have at most a 5% failure rate initially, and ideally <1% longer term. | * Therefore as a rough guide a new platform/testsuite must have at most a 5% per job failure rate initially, and ideally <1% longer term. | ||
* However, sheriffs will make the final determination of whether a job type has too many intermittent failures. This will be a based on a combination of factors including failure rate, length of time the failures have been occurring, owner interest in fixing them & whether TBPL is able to make bug suggestions. | * However, sheriffs will make the final determination of whether a job type has too many intermittent failures. This will be a based on a combination of factors including failure rate, length of time the failures have been occurring, owner interest in fixing them & whether TBPL is able to make bug suggestions. | ||