B2G/QA/Automation/UI/Strategy: Difference between revisions
< B2G | QA | Automation | UI
Jump to navigation
Jump to search
(→Plan) |
(→Plan) |
||
| Line 24: | Line 24: | ||
* [[/Integration vs Acceptance|Separate automation concerns for Integration and Acceptance tests]] | * [[/Integration vs Acceptance|Separate automation concerns for Integration and Acceptance tests]] | ||
For | For Acceptance: | ||
* [[/Streamline Acceptance Execution|Streamline execution for increased responsiveness with less effort]] | * [[/Streamline Acceptance Execution|Streamline execution for increased responsiveness with less effort]] | ||
| Line 31: | Line 31: | ||
* [[/Increase UAT coverage|Increase coverage using Gaia Acceptance]] | * [[/Increase UAT coverage|Increase coverage using Gaia Acceptance]] | ||
For | For Integration: | ||
* [[/Assist with CI tests|Assist the functional teams by increasing coverage]] | * [[/Assist with CI tests|Assist the functional teams by increasing Gaia Integration coverage]] | ||
* [[/Port Gaia Integration Python| | * [[/Port Gaia Integration Python|Port Gaia Integration Python tests to JavaScript for wider ownership]] | ||
For Community: | For Community: | ||
* [[/Increase Community|Find and implement new approaches to increasing community involvement]] (plan TBD) | * [[/Increase Community|Find and implement new approaches to increasing community involvement]] (plan TBD) | ||
Revision as of 00:30, 13 February 2015
Team Mandates
- Establish visible subject matter expertise in automation
- Own, expand and maintain QA acceptance test automation
- Increase confidence in acceptance test automation
- Provide expert assistance to improve all phases of UI test coverage
- Catch problems quickly and effectively
- Increase community support for FxOS QA
- Increase quality of and confidence in the product
Challenges
- Automation systems and concerns are poorly documented
- Development and QA have different approaches and needs from UI testing
- UI tests have had no developer support, largely because of Python implementation
- Team has been blocked too much by cross-team dependencies
- Test results are hidden in Jenkins
- Test results have too many spurious failures to sheriff
- Test results only looked at a limited number of times per day
- Community paths poorly defined and advertised
Plan
For Acceptance:
- Streamline execution for increased responsiveness with less effort
- Document how we develop and execute Acceptance automation
- Develop Gaia Acceptance, based on Marionette JS
- Increase coverage using Gaia Acceptance
For Integration:
- Assist the functional teams by increasing Gaia Integration coverage
- Port Gaia Integration Python tests to JavaScript for wider ownership
For Community: