B2G/QA/Automation/UI/Strategy/Develop Gaia End to end

Objective

Create a harness on top of Marionette JS that executes and controls a device with the level of care and confidence appropriate to UATs, as established via GaiaTest Python.

Challenges Addressed

• UI tests have had no developer support, largely because of Python implementation

The Problem

UATs are currently written in Python on top of GaiaTest. While this has worked well for us, the organization in general is moving towards JavaScript.

This heavily limits our ability to get development to cooperate in reviewing or making changes to our UATs. While QA does own these, some assistance is helpful and Python is a blocker.

In addition, Views (page object equivalents) will be created first in JavaScript to handle CI testing. If we do not align, we will have to reimplement the same basic classes in Python. Further, by using common view objects, developers will effectively maintain the UATs on the way to keeping CI tests green.

The Solution

Analyze GaiaTest on Python and implement any missing pieces on top of Marionette JS and Mocha, as appropriate. Port existing tests to the new harness.

Timeline

Q1:

• Transition schedule/gates [Geo]
• Documentation of Gaia UI Test harness behavior [Geo]
• GaiaTest JS v1 implementation [Geo]
• Review and improvement if necessary of View pattern [John]
• Establish JS UAT backlog with existing UATs to port [Johan]
• Conduct Q1 Pilot Program (MarionetteJS runs Gij) [Martijn]
• Help fix Marionette JS on-device when available [Martijn]
• If viable in Q1, start porting UAT smoketests

Q2:

• Port existing tests not addressed in Q1

Risks

• UATs must be able to work on-device. Marionette JS is not currently stable on device.
• The gap between Python GaiaTest and Marionette JS is not fully known yet.