B2G/QA/Automation/UI/Strategy/Assist with Gaia Integration tests: Difference between revisions

← Older edit

B2G/QA/Automation/UI/Strategy/Assist with Gaia Integration tests (view source)

Revision as of 16:55, 9 April 2015

830 bytes added , 9 April 2015

→‎Timeline

Jdorlus

Confirmed users

40

edits

@@ Line 1: / Line 1: @@
 =Objective=
-Increase stability of product and further relationship with development by helping them establish an initial set of continuous integration tests.
+Increase stability of product and further relationship with development by helping them expand the suite of Gaia Integration tests.
 =Challenges Addressed=
@@ Line 7: / Line 7: @@
 * Development and QA have different approaches and needs from UI testing
 * Test results are hidden in Jenkins
 * Test results only looked at a limited number of times per day
+*  The product is too unstable for acceptance testing during development phases
 =The Problem=
-There aren't enough CI tests. Builds are constantly unstable.
+There aren't enough Gaia Integration tests. Builds are constantly unstable. Also, since Acceptance tests primarily benefit QA directly, the larger organization doesn't always see the benefit they get from us.
 =The Solution=
-Add more heavily-isolated CI tests covering any behavior possible. Don't worry about whether they're sufficient for acceptance.
+Add more Gaia Integration tests covering any behavior reasonable. Don't worry about whether the tests are sufficient for Acceptance. Offer help to functional teams whenever possible. Make sure our contributions are visible.
 =Timeline=
@@ Line 21: / Line 22: @@
 Q1
-* Division of existing Gaia Integration Python tests by functional team
+* {{done|Criteria document for picking Gaia Integration tests to automate [Johan]}} [https://bugzilla.mozilla.org/show_bug.cgi?id=1136779]
-* Propose/add GIP tests to functional team backlogs
+* {{done|Analysis of Acceptance tests for additional Gaia Integration possibilities [Johan]}} [https://bugzilla.mozilla.org/show_bug.cgi?id=1139921]
-* Analysis of UATs for GIJ possibilities
+* {{missed|Propose/add tests to functional team backlogs [John]}}
-* Propose/add to functional team backlogs
+Ongoing:
 * Work with functional teams to implement agreed-upon tests
+* After verifying via functional team QA leads, help add more tests
 =Risks=
-We can't handle ownership of all the CI tests for the entire product, and they need to be maintained with the code in the same checkin. That means development needs to continue to own the tests. Because of this, we need to make sure we've verified they want the tests before we add them.
+QA can't handle the weight of ownership of all the Gaia Integration tests for the entire product. They should be written by the domain experts anyway, and they need to be maintained with the code in the same checkin.
+That means functional teams need to continue to own their own tests, with QA assisting as part of their functional team membership. Part of ownership is determining how many tests can be maintained given available resources and scoping coverage accordingly.
+Because of this, we need to make sure we've verified functional teams want the tests before we add them.