QA/Execution/Web Testing/Automation/Selenium Grid

moz-grid-config

moz-grid-config is a project for managing our Selenium infrastructure. It includes the latest version of Selenium that we're using, and a mechanism for starting the hub/nodes. You can find documentation on moz-grid-config at the github project page.

Architecture

Hub

The Selenium Grid hub is running on a VM in our data-center. For setup and administration see our internal Mana page.

Nodes

We have several Mac minis running as Selenium nodes and 6 VMs running Windows nodes in our data-center.

Mac OS X

• selenium.qa.mtv2.mozilla.com (10.252.73.143)
  • Mac Mini; 2.7 Ghz Intel Core i7; 8 GB
  • Mac OS X 10.7.4
• selenium-3.qa.mtv2.mozilla.com (10.252.73.196)
  • Mac Mini; 2.4 Ghz Intel Core 2 Duo; 4 GB
  • Mac OS X 10.6.8
• selenium-4.qa.mtv2.mozilla.com (10.252.73.197)
  • Mac Mini; 2.4 Ghz Intel Core 2 Duo; 4 GB
  • Mac OS X 10.6.8
• selenium-5.qa.mtv2.mozilla.com (10.252.73.145)
  • Mac Mini; 2.66 Ghz Intel Core 2 Duo; 8 GB
  • Mac OS X 10.6.8
• selenium-7.qa.mtv2.mozilla.com (10.252.73.9)
  • Mac OS X 10.7.4, 2.5 GHz Intel Core i5; 4GB 1067 MHz DDR3

Windows

• selenium1.qa.scl3.mozilla.com (10.22.73.149)
  • Windows 7, 8 GB RAM, 2 cores, 150GB disk
• selenium2.qa.scl3.mozilla.com (10.22.73.150)
  • Windows 7, 8 GB RAM, 2 cores, 150GB disk
• selenium3.qa.scl3.mozilla.com (10.22.73.151)
  • Windows 7, 8 GB RAM, 2 cores, 150GB disk
• selenium4.qa.scl3.mozilla.com (10.22.73.152)
  • Windows 7, 8 GB RAM, 2 cores, 150GB disk
• selenium5.qa.scl3.mozilla.com (10.22.73.153)
  • Windows 7, 8 GB RAM, 2 cores, 150GB disk
• selenium6.qa.scl3.mozilla.com (10.22.73.154)
  • Windows 7, 8 GB RAM, 2 cores, 150GB disk

Operations

Hub

For details on administering hub operations see our internal Mana page.

Nodes

Starting Mac OS X nodes

Connect to the appropriate machine, and execute the following from a terminal (you can use SSH or VNC):

   cd ~/moz-grid-config
   ant launch-node

Starting the Windows nodes

We use the msysgit tool on Windows. Connect to the appropriate machine using Remote Desktop, start the MsysGit tool on the desktop, and run the following commands:

   cd ~/moz-grid-config
   ant -autoproxy launch-node

We need the -autoproxy helper because the these nodes run through a proxy.

Stopping nodes

Terminate the process using Ctrl+C.

Adding a Browser

Configuration

You will need to fork and clone from the moz-grid-config github repository and work on a new branch.

1. Add the browser, alias, and path to grid_configuration.yml.
2. Add the browser, binary and details to the JSON configuration files.
3. Submit a pull request to the main GitHub repository from your branch.

Hub/Nodes

1. Prepare Jenkins for shutdown - this will allow all currently running jobs to finish, but will not start any new jobs.
2. Once no jobs are running, stop the Selenium Grid hub
3. Pull in the latest changes from moz-grid-config
4. Start the Selenium Grid hub
5. For each node:
   1. Stop the Selenium server
   2. Pull in the latest changes from moz-grid-config
   3. Install the appropriate browser
   4. Start the Selenium server

Installing Firefox

• All new browsers should be installed with the version number in the path so that moz-grid-config can always find them.
• Install mozdownload and mozinstall using pip to download and install the Firefox binaries.
  • Use --help for both mozdownload and mozinstall to see all available options
  • To download the latest release using mozdownload you can simply use: mozdownload -v latest
  • To download the latest Firefox Nightly build using mozdownload you can use: mozdownload -t daily --branch=mozilla-central
  • To download the latest Firefox Aurora build using mozdownload you can use: mozdownload -t daily --branch=mozilla-aurora
  • Releases are also available to download manually here

Mac OS X

From a terminal:

   mozdownload -v 14.0.1 -d /tmp
   mozinstall -d /Applications/ /tmp/firefox-14.0.1.en-US.mac.dmg
   mv /Applications/Firefox.app/ "/Applications/Firefox 14.app/"
   rm /tmp/firefox-14.0.1.en-US.mac.dmg

Windows

From a command prompt:

   mozdownload -v 14.0.1 -d %TMP%
   mozinstall -d %USERPROFILE% %TMP%\firefox-14.0.1.en-US.win32.exe
   move %USERPROFILE%/firefox-14 "\Program Files\Mozilla Firefox 14"
   del %TMP%\firefox-14.0.1.en-US.win32.exe

Setting hostname on Mac OS X

Shamelessly copied from http://budporter.net/?p=47

If you are running a version of OSX prior to 10.5, then look in /etc/hostconfig for the HOSTNAME= parameter and set it to be the FQDN (Fully Qualified Domain Name) that you want to use. So, for host foo located in the domain bar.com, the entry would be as follows:

   HOSTNAME=foo.bar.com

For OSX 10.5, /etc/hostconfig is being deprecated. There are a couple of preferred methods to do this in 10.5:

   sudo hostname -s foo.bar.com

or:

   sudo scutil –set HostName foo.bar.com

You can verify the change by issuing the following command:

   hostname

The output will display the FQDN of the computer.

Updating the Grid (step by step)

Before updating the Grid we need to test locally with the latest Firefox and Selenium versions.

1. You will need to fork and clone from the moz-grid-config github repository and work on a new branch (latest-version) (see Configuration.
   • Install the latest Firefox version on your local machine
   • Change the Selenium version to be the latest (in default.properties file), then it will be downloaded automatically when starting the hub/node.
   • Update the JSON files for your operating system - add the latest browser, binary and details. (for both hub and nodes)
   • Update the Selenium client using "pip install -r requirements.txt". Start the hub and node and start testing.
2. After changes are merged into the main GitHub repository, move the main Grid to run on "latest-version" branch.
3. Once testing on the main Grid instance is successful, merge the latest-version branch into master and switch back to the master branch on the hub and nodes.

References

• Jenkins dashboard
• Selenium Grid console
• moz-grid-config on github