B2G/QA/Automation/UI/Strategy/Assist with Gaia Integration tests

Objective

Increase stability of product and further relationship with development by helping them expand the initial set of Gaia Integration tests.

Challenges Addressed

• Development and QA have different approaches and needs from UI testing
• Test results are hidden in Jenkins
• Test results only looked at a limited number of times per day
• The product is too unstable for acceptance testing during development phases

The Problem

There aren't enough Gaia Integration tests. Builds are constantly unstable. Also, since Acceptance tests primarily benefit QA directly, the larger organization doesn't always see the benefit they get from us. Helping functional teams out would increase visibility.

The Solution

Add more Gaia Integration tests covering any behavior reasonable. Don't worry about whether the tests are sufficient for Acceptance. Offer help to functional teams whenever possible.

Timeline

Q1

• Criteria document for picking Gaia Integration tests to automate [Johan]
• Analysis of Acceptance tests for additional Gaia Integration possibilities [Johan]
• Propose/add tests to functional team backlogs [John]
• Work with functional teams to implement agreed-upon tests
• After verifying via functional team QA leads, help add more tests

Risks

QA can't handle the weight of ownership of all the Gaia Integration tests for the entire product. They should be written by the domain experts anyway, and they need to be maintained with the code in the same checkin.

That means functional teams need to continue to own their own tests, with QA assisting as part of their functional team membership. Part of ownership is determining how many tests can be maintained given available resources and scoping coverage accordingly.

Because of this, we need to make sure we've verified functional teams want the tests before we add them.