Auto-tools/Projects/JetPerf

From MozillaWiki
< Auto-tools‎ | Projects
Revision as of 17:32, 28 January 2012 by Jhammel (talk | contribs) (→‎Overview: link to bugs)
Jump to navigation Jump to search

Jetpack Performance Tests

Running talos ts (startup) tests on Firefox for the addon builder set of addons

Overview

The Jetpack team maintains a set of APIs for add-ons to use and want to see how use of those APIs affect performance. They will provide a set of simple add-ons that use these APIs for testing. In the past we have brought up a set of unittests for jetpack on tinderbox. Recently there has been work to test startup performance of addons on the AMO website.

Tracking bug: https://bugzilla.mozilla.org/show_bug.cgi?id=717036

Bugs to be labeled with the jetpack+talos whiteboard entry

Goals

  • Have a Tjetpack test which runs on checkins (and try server) to detect performance issues
  • Track delta of each addon and make it graphable

Status

Jetpack weekly meetings Tuesdays @ 10:00am Pacific Time

Contacts

  • Ateam: jmaher, jhammel
  • Jetpack: mossop, warner
  • Releng: lsblakk

Plan of Action

  • get list of addons from jetpack team
  • get a list of commands to obtain the addons (checkout and build)
  • create a Tjetpack test in talos
  • proof of concept run to get results (by hand)
  • determine graph server requirements (if any)
  • review work with Jetpack team, ensure this meets their needs/goals
  • integrate into talos
  • staging and deploy talos.

Problems to resolve

  • figure out how to integrate the download and compilation of jetpack addons
    • this is in the jetpack unittests, but we have much less flexibility in our talos suite
  • how to run all the permutations we care about (most likely 3 different tests):
    • baseline + addon1
    • baseline + addon2
    • baseline + addon3
  • how to display the results properly?
    • is ts_addon1 data point good?
    • is ts_addon2 ok?
  • which branches should this run on?

Links