canmove, Confirmed users
2,041
edits
QA/Automation/Projects/Mozmill Automation/On Demand Test Framework: Difference between revisions
Jump to navigation
Jump to search
QA/Automation/Projects/Mozmill Automation/On Demand Test Framework (view source)
Revision as of 21:22, 16 November 2011
, 16 November 2011no edit summary
No edit summary |
|||
| Line 34: | Line 34: | ||
| Implementation of a remote mechanism to trigger Mozmill functional and update tests for Firefox releases | | Implementation of a remote mechanism to trigger Mozmill functional and update tests for Firefox releases | ||
|} | |} | ||
==Requirements== | |||
* We want to be able to kick off test runs with as little interaction and micromanaging as we can. | |||
* A test run equates to a script with one or more parameters, which will be run on one or more environments. These parameters correspond mostly to the test matrix, and will vary per run type. | |||
* The test run is capable of, within a given environment, doing everything from app deployment to testing to app cleanup. Or we can split these up if more convenient. | |||
* We'll want the system to spread testing among a number of available machines per environment, and ultimately create the machines as needed. | |||
* Ultimately, we'd like the ability to slice the test run on a parameter value to run in parallel. If we can get it, understanding how to slice the test run per-test would be even better. | |||
* We'd like to clearly monitor run progress, and watchdog for hung machines. | |||
* We'd like to see understandable test results, preferably consolidated across a whole run (at least per-env, if not cross-env). | |||
==Project Milestones== | ==Project Milestones== | ||