|
|
| Line 31: |
Line 31: |
| =Resources= | | =Resources= |
|
| |
|
| =To move= | | =TBD= |
| [https://docs.google.com/Doc?id=d9cz3wk_17hmkx6p2w Google Document]
| |
| [https://bugzilla.mozilla.org/show_bug.cgi?id=564648 Bug 564648] | | [https://bugzilla.mozilla.org/show_bug.cgi?id=564648 Bug 564648] |
|
| |
| =Project Details=
| |
| Only running Mozmill tests on buildbot or on the release-test machine in the QA lab is not sufficient to find misbehavior's of Firefox. A more wide-spread test-run is necessary. Such a possibility will be the result of the crowd testing project. An extension has to be developed which will automatically prepare the system, executes the test-run, and send the results to our report server for analysis.
| |
|
| |
| ==The Extension==
| |
| The easiest way to add additional features to Firefox is to install an extension. The same way will be used for this project. The extension, which will be offered on [https://addons.mozilla.org/ addons.mozilla.org], will add it's own toolbar right below of the existing ones. With a dropdown field the user will be able to select the wanted test-run, e.g. normal tests, update tests, or l10n tests. By clicking the 'Run' button the test-run gets executed.
| |
|
| |
| ==Preparing the System==
| |
| To run Mozmill tests, the system has to be prepared to match a specific set of requirements. [https://developer.mozilla.org/en/Mozmill#The_Command_Line_Client The steps] which are necessary aren't the easiest ones and can cause trouble, especially when you are doing it for the first time.
| |
|
| |
| That's why the extension has to create a virtual environment, which will contain all needed applications and tools. The following steps will be necessary:
| |
|
| |
| * Retrieving the Python interpreter (or bundle it with the extension)
| |
| * Create a virtual Python environment
| |
| * Install the latest release of Mozmill
| |
| * Retrieve the automation scripts
| |
|
| |
| ==Executing Mozmill Tests==
| |
| Given by the selected dropdown entry on the toolbar the Python script of the related test-run gets executed. It will perform the following actions:
| |
|
| |
| * Cloning the Mozmill tests from the [http://hg.mozilla.org/qa/mozmill-tests/ mozmill-tests repository]
| |
| * Updating the tests branch to the Gecko branch of the currently running version of Firefox
| |
| * Starting another instance of Firefox
| |
| * Execute all existent tests
| |
|
| |
| ==Results Analysis==
| |
| Once a test-run has been finished the results can be uploaded to our [http://brasstacks.mozilla.com/couchdb/mozmill/_design/reports/_list/summary/summary report server]. There they can be examined and checked for new failures. A [https://wiki.mozilla.org/QA/Mozmill_Test_Automation/Dashboard web dashboard] will be created to offer the most valuable information.
| |