Auto-tools/Projects/JetPerf

Jetpack Performance Tests

Running talos ts (startup) tests on Firefox for the addon builder set of addons

Overview

The Jetpack team has a set of addons that the maintain and want to see how those addons affect performance. In the past we have brought up a set of unittests for jetpack on tinderbox. Recently there has been work to test startup performance of addons on the AMO website.

Goals

• Have a Tjetpack test which runs on checkins (and try server) to detect performance issues
• Track delta of each addon and make it graphable

Plan of Action

• get list of addons from jetpack team
• get a list of commands to obtain the addons (checkout and build)
• create a Tjetpack test in talos
• proof of concept run to get results (by hand)
• determine graph server requirements (if any)
• review work with Jetpack team, ensure this meets their needs/goals
• integrate into talos
• staging and deploy talos.

Problems to resolve

• figure out how to integrate the download and compilation of jetpack addons
  • this is in the jetpack unittests, but we have much less flexibility in our talos suite

• how to run all the permutations we care about (most likely 3 different tests):
  • baseline + addon1
  • baseline + addon2
  • baseline + addon3

• how to display the results properly?
  • is ts_addon1 data point good?
  • is ts_addon2 ok?

• which branches should this run on?

Contacts

• Ateam: jmaher
• Jetpack: