Auto-tools/Projects/m21s

Introduction

m21s is the code name of the mozmill-marionette-e10s project, the project whose aim is to replace mozmill with e10s-compliant marionette tests. The project consists roughly of the following pieces:

• adapt marionette to e10s
• add features to marionette required by mozmill tests
• rewrite existing mozmill tests as marionette tests
• update the mozmill infrastructure and CI to utilize the new tests and harness

Many of these pieces can be done in parallel, but this is a significant project which will likely take multiple months to fully deliver.

Rationale

Mozmill is currently used for release and update testing by QA. It provides coverage not duplicated elsewhere, and sometimes finds important bugs. Mozmill was written long before e10s and features a model which includes promiscuous chrome-content interactions implemented in a way which is not trivial to port to an e10s-compatible model. Making mozmill e10s-compatible would involve updating the harness, the libraries and the test themselves.

Although this is achievable, there has been momentum building for the idea of migrating mozmill on top of marionette, to benefit from the stability of marionette's protocol. An evaluation of this idea has been performed, and it was determined that this solution would be difficult to implement and likely problematic as well. Combined with the requirement of e10s-compatibility, this idea was determined to be not worth the effort and has been abandoned in favor of simply rewriting all of the existing mozmill tests as marionette tests, after the necessary feature work to marionette has been accomplished.

Team

• Jonathan Griffin (jgriffin) - project management
• Henrik Skupin (whimboo) - mozmill maintainer and infrastructure/CI owner
• David Burns (AutomatedTester) - Marionette lead
• Andrew Halberstadt (ahal) - automation developer
• Chris Manchester (chmanchester) - automation developer
• Andreas Tolfsen (ato) - automation developer

Requirements

Marionette enhancements

1. e10s support for Marionette, bug 1046512, [chmanchester]
2. modal dialog support, bug 906712, [ato]
3. restart capability, including passing est runtime data between processes, bug 940954, [AutomatedTester]
4. anonymous node support in chrome, bug 1080764, [ahal]
5. be able to use l10n lookups; needs further investigation [chmanchester]
6. be able to install and work with addons (may need minor runner changes), [owner TBD]
7. be able to report results to mozmill dashboard (add a runner mixin?), [owner TBD]

Mozmill infrastructure and CI changes

1. rewrite mozmill automation scripts ( https://github.com/mozilla/mozmill-automation/ ), [whimboo]
2. update the pre-configured environments ( https://github.com/whimboo/mozmill-environment ), [whimboo]
3. update mozmill-ci to handle marionette instead of mozmill, [whimboo]
4. update the mozmill-crowd add-on to let community still participate in testing, [whimboo]
5. revise documentation, [whimboo]
6. rework Mozmill tests for TPS to become Marionette tests (probably low priority), [whimboo]

Test conversion

We will convert a small number of tests that don't rely on outstanding marionette enhancements, as a POC.

The plan for conversion of the remaining tests is still TBD.

Bugs

Open bugs

No results.

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);