QA/Automation/Projects/Mozmill Automation/Mozmill CI

From MozillaWiki
Jump to navigation Jump to search

Overview

The Mozmill Automation team currently performs Mozmill tests triggered by a daily cron job. This approach has some weaknesses:

  • If no build lands, we still attempt to perform Mozmill tests
  • If a build lands off schedule, we attempt to test it at the wrong time or miss it completely
  • There's no way to use this style of trigger to pre-qualify a build (or even promptly post-qualify it) so results have limited impact
  • Because we update build-over-build, using the same installation, anomalies in testing (or bugs in a given build) can cause the process to get "off-track" and poison subsequent Mozmill tests.

Instead, we'd like to move towards performing Mozmill tests immediately when a build has been made available. We'd also like to independently download and verify each day's build, rather than relying on using the same install day over day.

The ultimate goal is to build a system within QA, then hand off the process to Release Engineering for ongoing maintenance.

Name: Triggered Daily Test-runs
Leads: Henrik Skupin, Dave Hunt
Contributors: Geo Mealer
Tracker: https://www.pivotaltracker.com/projects/298905?label=Triggered%20Tests
Tracking Bug: https://bugzilla.mozilla.org/show_bug.cgi?id=709052
Repository: http://hg.mozilla.org/qa/mozmill-automation/
Etherpad: http://etherpad.mozilla.com:9000/qas-triggered-tests
Retrieved from "https://wiki.allizom.org/index.php?title=QA/Automation/Projects/Mozmill_Automation/Mozmill_CI&oldid=377293"