B2G/QA/Automation/UI/Strategy/Assist with Gaia Integration tests

Objective

Increase stability of product and further relationship with development by helping them establish an initial set of continuous integration tests.

Challenges Addressed

• Development and QA have different approaches and needs from UI testing
• Test results are hidden in Jenkins
• Test results only looked at a limited number of times per day

The Problem

There aren't enough CI tests. Builds are constantly unstable. Also, since UATs primarily benefit QA directly, development doesn't always see the benefit they get from us. Helping them out would increase visibility.

The Solution

Add more heavily-isolated CI tests covering any behavior possible. Don't worry about whether they're sufficient for acceptance. Offer help to functional teams whenever possible.

Timeline

Q1

• Criteria document for picking Gaia Integration tests to automate [Johan]
• Analysis of UATs for additional Gaia Integration possibilities [Johan]
• Propose/add to functional team backlogs [John]
• Work with functional teams to implement agreed-upon tests
• After verifying via functional team QA leads, help add more tests

Risks

We can't handle the weight of ownership of all the CI tests for the entire product. They should be written by the domain experts anyway, and they need to be maintained with the code in the same checkin.

That means functional teams need to continue to own their own tests. Because of this, we need to make sure we've verified they want the tests before we add them.