QA/Mozmill Test Automation/On Demand Provisioning

Overview

Lead:	Al Billings
Dates:	Plan formalized by end of 2011Q1
Status:	Writing initial plan and doing investigation
Documentation:	TBD

Summary

The QA organization needs on demand resources for a variety of client testing with Firefox.

One goal is to be able to set up a client machine on demand with specific combinations of anti-virus software, firewall software, extensions, or plugins and run Firefox builds with the Mozmill Crowd extension.

Being able to provision multiple machines with unique configurations on demand will allow QA to test specific scenarios that are currently not tested.

Project Plan

Milestones

Milestone 1

Completion of:

• something
• something

Milestone 2

Completion of:

• something
• something

Milestone 3

Completion of:

• something
• something

Milestone 4

Completion of:

• something
• something

Notes

VM management

• Host-based
  • Vmware ESX
    • Already supported by Mozilla IT using HP blades servers.
    • IT says it may have two outdated servers in the San Jose data-center that we can use.
    • IT also says that there is an existing QA license for ESX (still waiting for details).
    • According to VMware, is superceded in future by shiny new ESXi, which doesn't have same feature set. Need to know Mozilla's migration path, if any, to ESXi.
    • Pricing for ESX vs. ESXi.
    • Supports all of the Windows and Linux operating systems we currently use.
    • supports scriptable installations through utilities like KickStart.
    • supports managing VMs through VI Web Access.
    • uses vcenter converter to convert existing Vmware VMs or hardware machines to ESX capable VMs.
    • Scriptable via vSphere Command-Line Interface (VCLI) (Though this may only apply to ESXi and ESX may use a different set of local tools. Unclear.)
    • vSphere Management Assistant (vMA) - prepackaged Linux VM that can deploy scripts and third-party agents to manage ESX systems.
  • Xen
    • Runs as host under:
      • SUSE Linux Enterprise 10
      • RHEL 5/Fedora 7
      • Sun Solaris
      • Debian's Etch
    • Client, guest, VMs:
      • Linux
      • OpenBSD, FreeBSD
      • Windows (if the host machine's processor supports hardware virtualization)
    • Commercial versions
      • Citrix XenServer
• Cloud-based
  • Amazon Elastic Cloud Compute (EC2)
    • Supports:
      • Windows Server 2003/2008
      • Fedora, Gentoo, Ubuntu, SUSE, Debian
    • No Windows client systems (XP, Vista, 7)
    • Can import VMware VMs for Windows Server 2008 SP2
    • Scriptable through command line interface of Rain Toolkit
    • Pricing:
      • Machines cost between $0.085 to $0.68 an hour for Linux, $0.12 to $0.96 an hour for Windows
      • Data transfer in or out is $0.10 per GB.
      • $0.11 per GB-month of provisioned storage, $0.11 per 1 million I/O requests
    • Primary advantage is immediate, on-demand, infinite* scaling based on ability to pay.
  • Rackspace
    • Runs VMware-based virtual machines in cloud using either ESX or ESXi
    • Appears to only suppport Linux or Windows Server systems.
• Other Hosting Options
  • Mac OS X would have to be run on its own Apple hardware due to licensing.
  • One option is scriptable, imaging software and not use VMs at all.
    • Clonezilla
• Execution/Content management
  • Use Mozmill crowd extension with BYOB technology:
    • Combinations of:
      • anti-virus software
      • firewall versions
      • OS versions
      • browser plugins
      • extensions - top 100?
      • Reporting

• Action items
  • Expand data for Cloud automation and Mozmill Server above
    • identify
      • dependencies
      • milestones
      • components