QA/Automation/Projects/Addons/MozmillCrowd: Difference between revisions

From MozillaWiki
< QA‎ | Automation‎ | Projects‎ | Addons
Jump to navigation Jump to search
 
(12 intermediate revisions by the same user not shown)
Line 1: Line 1:
__NOTOC__
__NOTOC__
=Overview=
==Overview==
{|  
Running Mozmill tests should be as easy as possible and doable by any interested tester. No complicated setup routines or pre-requisites should stop anyone by successfully running our existing Mozmill tests in their Firefox browser. As result a rich set of results, which will be collected from testers world-wide, can be collected and used to nail down regressions or to find problems related to the geographical position or installed locale of Firefox.
  | style="width: 45%" |
 
  | style="width: 55%" |
{| style="width: 100%; margin-bottom: 2em" |
  | style="width: 25%" |
  | style="width: 75%" |
  |- valign="top"
  |- valign="top"
  | '''Project:'''
  | '''Name:'''
  | Mozmill-Crowd Extension
  | Mozmill Crowd - Crowd-sourced functional Testing with Mozmill
  |- valign="top"
  |- valign="top"
  | '''Leads:'''
  | '''Leads:'''
Line 13: Line 15:
  | '''Team Assistance:'''
  | '''Team Assistance:'''
  | n/a
  | n/a
|- valign="top"
| '''Tracker:'''
| https://www.pivotaltracker.com/projects/285911
|}
|}


Running Mozmill tests should be as easy as possible and doable by any interested tester. No complicated setup routines or pre-requisites should stop anyone by successfully running our existing Mozmill tests in their Firefox browser. As result a rich set of results, which will be collected from testers world-wide, can be collected and used to nail down regressions or to find problems related to the geographical position or installed locale of Firefox.
==Platform==
 
* [[/Documentation|Documentation]]
=Mozmill-Crowd Platform=
* [[QA/Mozmill_Test_Automation/Crowd_Testing/Documentation|Documentation]]
* [http://etherpad.mozilla.com:9000/qa-auto-mozmill-crowd Etherpad] for brainstorming and discussions
* [http://etherpad.mozilla.com:9000/qa-auto-mozmill-crowd Etherpad] for brainstorming and discussions
* [[QA/Mozmill_Test_Automation/Crowd_Testing/FAQ|FAQ]]
* [[/FAQ|FAQ]]
* [[QA/Mozmill_Test_Automation/Crowd_Testing/Presentations|Presentations]]
* [[/Presentations|Presentations]]
 
=Mozmill-Crowd Production=
* [[QA/Mozmill_Test_Automation/Crowd_Testing/Roadmap|Product Roadmap]]
* [[QA/Mozmill_Test_Automation/Crowd_Testing/Release|Release Process]]
* [[QA/Mozmill_Test_Automation/Crowd_Testing/Schedule|Release Schedule]]
* [[QA/Mozmill_Test_Automation/Crowd_Testing/Contribution|How to Contribute]]
 
=Resources=
 
=To move=
[https://docs.google.com/Doc?id=d9cz3wk_17hmkx6p2w Google Document]
[https://bugzilla.mozilla.org/show_bug.cgi?id=564648 Bug 564648]
 
=Project Details=
Only running Mozmill tests on buildbot or on the release-test machine in the QA lab is not sufficient to find misbehavior's of Firefox. A more wide-spread test-run is necessary. Such a possibility will be the result of the crowd testing project. An extension has to be developed which will automatically prepare the system, executes the test-run, and send the results to our report server for analysis.
 
==The Extension==
The easiest way to add additional features to Firefox is to install an extension. The same way will be used for this project. The extension, which will be offered on [https://addons.mozilla.org/ addons.mozilla.org], will add it's own toolbar right below of the existing ones. With a dropdown field the user will be able to select the wanted test-run, e.g. normal tests, update tests, or l10n tests. By clicking the 'Run' button the test-run gets executed.
 
==Preparing the System==
To run Mozmill tests, the system has to be prepared to match a specific set of requirements. [https://developer.mozilla.org/en/Mozmill#The_Command_Line_Client The steps] which are necessary aren't the easiest ones and can cause trouble, especially when you are doing it for the first time.
 
That's why the extension has to create a virtual environment, which will contain all needed applications and tools. The following steps will be necessary:
 
* Retrieving the Python interpreter (or bundle it with the extension)
* Create a virtual Python environment
* Install the latest release of Mozmill
* Retrieve the automation scripts
 
==Executing Mozmill Tests==
Given by the selected dropdown entry on the toolbar the Python script of the related test-run gets executed. It will perform the following actions:


* Cloning the Mozmill tests from the [http://hg.mozilla.org/qa/mozmill-tests/ mozmill-tests repository]
==Production==
* Updating the tests branch to the Gecko branch of the currently running version of Firefox
* [[/Roadmap|Product Roadmap]]
* Starting another instance of Firefox
* [[/Release|Release Process]]
* Execute all existent tests
* [[/Contribution|How to Contribute]]


==Results Analysis==
==Resources==
Once a test-run has been finished the results can be uploaded to our [http://brasstacks.mozilla.com/couchdb/mozmill/_design/reports/_list/summary/summary report server]. There they can be examined and checked for new failures. A [https://wiki.mozilla.org/QA/Mozmill_Test_Automation/Dashboard web dashboard] will be created to offer the most valuable information.

Latest revision as of 10:40, 13 January 2014

Overview

Running Mozmill tests should be as easy as possible and doable by any interested tester. No complicated setup routines or pre-requisites should stop anyone by successfully running our existing Mozmill tests in their Firefox browser. As result a rich set of results, which will be collected from testers world-wide, can be collected and used to nail down regressions or to find problems related to the geographical position or installed locale of Firefox.

Name: Mozmill Crowd - Crowd-sourced functional Testing with Mozmill
Leads: Henrik Skupin, Aaron Train
Team Assistance: n/a
Tracker: https://www.pivotaltracker.com/projects/285911

Platform

Production

Resources

Retrieved from "https://wiki.allizom.org/index.php?title=QA/Automation/Projects/Addons/MozmillCrowd&oldid=887689"