QA/Automation/Projects/Endurance Tests/Documentation

Introduction

The Endurance Tests are intended for replicating and discovering issues related to degradation of application performance over time. This is achieved by running a snippet of test code repeatedly, gathering details on system resources, and reporting these metrics for review. Check out the project page for more details on Endurance Tests.

Running Endurance Tests

To run Endurance Tests it is necessary to clone Mozmill's Automation Testruns repository and run the testrun_endurance.py script. Run this with the flag --help to see a list of command line options:

./testrun_endurance.py --help
Usage: testrun_endurance.py [options] (binary|folder)

Options:
  --version                 show program's version number and exit
  -h, --help                show this help message and exit
  --delay=DELAY             Duration (in seconds) to wait before each iteration
  --iterations=ITERATIONS   Number of times to repeat each test snippet
  --logfile=PATH            Path to the log file
  -r URL, --report=URL      Send results to the report server
  --repository=URL          URL of a custom remote or local repository

Writing Endurance Tests

All Endurance Tests need to instantiate an EnduranceManager from the endurance.js shared-module. The only other requirement is that the test calls the run method, with a single argument. This argument is the function that will be repeated for each iteration. An example endurance test template is below:

var Endurance = require("../../../shared-modules/endurance");

function setupModule() {
  controller = mozmill.getBrowserController();

  enduranceManager = new Endurance.EnduranceManager(controller);
}

function testExample() {
  enduranceManager.run(function () {
    // Action 1
    enduranceManager.addCheckpoint("Action 1");
    // Action 2
    enduranceManager.addCheckpoint("Action 2");
  });
}

Reports

Currently the Mozmill Test Results Dashboard does not support the Endurance Tests results, however there is a work in progress version with support for these reports. In order to use this, set the --report command line option to http://davehunt.couchone.com/mozmill. You can then view the reports at http://mozmill.blargon7.com/#/endurance/reports