Auto-tools/Projects/JetPerf

< Auto-tools‎ | Projects
Revision as of 17:32, 28 January 2012 by Jhammel (talk | contribs) (→‎Overview: link to bugs)

Jetpack Performance Tests

Running talos ts (startup) tests on Firefox for the addon builder set of addons

Overview

The Jetpack team maintains a set of APIs for add-ons to use and want to see how use of those APIs affect performance. They will provide a set of simple add-ons that use these APIs for testing. In the past we have brought up a set of unittests for jetpack on tinderbox. Recently there has been work to test startup performance of addons on the AMO website.

Tracking bug: https://bugzilla.mozilla.org/show_bug.cgi?id=717036

Bugs to be labeled with the jetpack+talos whiteboard entry

Goals

  • Have a Tjetpack test which runs on checkins (and try server) to detect performance issues
  • Track delta of each addon and make it graphable

Status

Jetpack weekly meetings Tuesdays @ 10:00am Pacific Time

Contacts

  • Ateam: jmaher, jhammel
  • Jetpack: mossop, warner
  • Releng: lsblakk

Plan of Action

  • get list of addons from jetpack team
  • get a list of commands to obtain the addons (checkout and build)
  • create a Tjetpack test in talos
  • proof of concept run to get results (by hand)
  • determine graph server requirements (if any)
  • review work with Jetpack team, ensure this meets their needs/goals
  • integrate into talos
  • staging and deploy talos.

Problems to resolve

  • figure out how to integrate the download and compilation of jetpack addons
    • this is in the jetpack unittests, but we have much less flexibility in our talos suite
  • how to run all the permutations we care about (most likely 3 different tests):
    • baseline + addon1
    • baseline + addon2
    • baseline + addon3
  • how to display the results properly?
    • is ts_addon1 data point good?
    • is ts_addon2 ok?
  • which branches should this run on?

Links