QA/Automation/Projects/Mozmill Automation/Mozmill CI: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
Line 1: Line 1:
__NOTOC__
__NOTOC__
=Overview=
=Overview=
The Mozmill Automation team currently performs Mozmill tests triggered by a daily cron job. This approach has some weaknesses:
The Automation Development team is working on a Mozmill CI solution which triggers Mozmill tests immediately when a new build of Firefox has been made available.
* If no build lands, we still attempt to perform Mozmill tests
* If a build lands off schedule, we attempt to test it at the wrong time or miss it completely
* There's no way to use this style of trigger to pre-qualify a build (or even promptly post-qualify it) so results have limited impact
* Because we update build-over-build, using the same installation, anomalies in testing (or bugs in a given build) can cause the process to get "off-track" and poison subsequent Mozmill tests.
 
Instead, we'd like to move towards performing Mozmill tests immediately when a build has been made available. We'd also like to independently download and verify each day's build, rather than relying on using the same install day over day.


The ultimate goal is to build a system within QA, then hand off the process to Release Engineering for ongoing maintenance.
The ultimate goal is to build a system within QA, then hand off the process to Release Engineering for ongoing maintenance.
Line 22: Line 16:
  |- valign="top"
  |- valign="top"
  | '''Contributors:'''
  | '''Contributors:'''
  | Geo Mealer
  | n/a
|- valign="top"
| '''Tracker:'''
| https://www.pivotaltracker.com/projects/298905?label=Triggered%20Daily%20Tests
|- valign="top"
| '''Tracking Bug:'''
| https://bugzilla.mozilla.org/show_bug.cgi?id=709052
  |- valign="top"
  |- valign="top"
  | '''Repository:'''  
  | '''Repository:'''  
  | http://hg.mozilla.org/qa/mozmill-automation/
  | https://github.com/mozilla/mozmill-ci
  |- valign="top"
  |- valign="top"
  | '''Etherpad:'''
  | '''Etherpad:'''
  | http://etherpad.mozilla.com:9000/qas-triggered-tests
  | https://etherpad.mozilla.org/mozmill-ci
|}
|}


Revision as of 15:51, 23 May 2012

Overview

The Automation Development team is working on a Mozmill CI solution which triggers Mozmill tests immediately when a new build of Firefox has been made available.

The ultimate goal is to build a system within QA, then hand off the process to Release Engineering for ongoing maintenance.

Name: Triggered Daily Test-runs
Leads: Henrik Skupin, Dave Hunt
Contributors: n/a
Repository: https://github.com/mozilla/mozmill-ci
Etherpad: https://etherpad.mozilla.org/mozmill-ci

Documentation

A complete set of documents can be found on the pages linked below:

Roadmap

Date Version Details
Dec 12th, 2011 - Documentation ready for CI implementation
Dec 31st, 2011 - CI system running for daily tests
Retrieved from "https://wiki.allizom.org/index.php?title=QA/Automation/Projects/Mozmill_Automation/Mozmill_CI&oldid=433731"