QA/Automation/Projects/Addons/MozmillCrowd
Overview
| Lead: | Henrik Skupin |
| Co-workers: | n/a (help wanted) |
| Dates: | v0.1 by end of Q2/10 |
| Status: | Collecting requirements |
| Repository Location: | http://github.com/whimboo/mozmill-cloud |
| Tracking Bug: | Bug 564648 |
Excerpt
Running Mozmill tests should be as easy as possible and doable by any interested party. No complicated setup routines or pre-requisites should stop anyone by successfully running our existing Mozmill tests in their Firefox browser. As result a rich set of results, which will be collected from testers world-wide, will be stored on our results server and used to nail down regressions or to find problems related to your geographical position or installed language pack.
Project Details
Only running Mozmill tests on buildbot or on the release-test machine in the QA lab is not sufficient to find misbehavior's of Firefox. A more wide-spread test-run is necessary. Such a possibility will be the result of the cloud testing project. An extension has to be developed which will automatically prepare the system, executes the test-run, and send the results to our report server for analysis.
The Extension
The easiest way to add additional features to Firefox is to install an extension. The same way will be used for this project. The extension, which will be offered on addons.mozilla.org, will add it's own toolbar right below of the existing ones. With a dropdown field the user will be able to select the wanted test-run, e.g. normal tests, update tests, or l10n tests. By clicking the 'Run' button the test-run gets executed.
Preparing the System
To run Mozmill tests, the system has to be prepared to match a specific set of requirements. The steps which are necessary aren't the easiest ones and can cause trouble, especially when you are doing it for the first time.
That's why the extension has to create a virtual environment, which will contain all needed applications and tools. The following steps will be necessary:
- Retrieving the Python interpreter (or bundle it with the extension)
- Create a virtual Python environment
- Install the latest release of Mozmill
- Retrieve the automation scripts
Executing Mozmill Tests
Given by the selected dropdown entry on the toolbar the Python script of the related test-run gets executed. It will perform the following actions:
- Cloning the Mozmill tests from the mozmill-tests repository
- Updating the tests branch to the Gecko branch of the currently running version of Firefox
- Starting another instance of Firefox
- Execute all existent tests
Results Analysis
Once a test-run has been finished the results can be uploaded to our report server. There they can be examined and checked for new failures. A web dashboard will be created to offer the most valuable information.