ReleaseEngineering/How To/Validate a New Config: Difference between revisions

← Older edit

ReleaseEngineering/How To/Validate a New Config (view source)

Revision as of 07:49, 30 August 2019

No change in size , 30 August 2019

no edit summary

Stephend

Confirmed users

9,511

edits

@@ Line 9: / Line 9: @@
 * etc.
-We often run into this scenario at least once/quarter and in conversations this is usually referred to as "greening up the tests".  This is intended to document that process.
+We often run into this scenario at least once/quarter and in conversations this is usually referred to as "greening up the tests."  This is intended to document that process.
 The scope here is to assume you are testing a single branch, a single platform or ideally a single build type/config, and will need to test most if not all the tests.
@@ Line 26: / Line 26: @@
 *** file a bug, needinfo someone to help get a resolution
-typically this process results in dozens of bugs and often many test failures are the same root cause.  It is a good idea to pick the top set of failures at first that seem to be common issues (installer, process/crash, reftest and fonts, canvas/webgl, etc.) and get bugs on file with examples of various tests.  Having traction on those bugs will often result in fixing many more.
+Typically this process results in dozens of bugs and often many test failures are the same root cause.  It is a good idea to pick the top set of failures at first that seem to be common issues (installer, process/crash, reftest and fonts, canvas/webgl, etc.) and get bugs on file with examples of various tests.  Having traction on those bugs will often result in fixing many more.
 One word of caution here is that the browser is a changing environment every day.  We change features, fix things, break things, and most importantly our tests are changing daily as well.  So what failures you found yesterday might not be the same failures you see today.  Typically I view this as a 3 pass process: