Firefox OS/Performance/Boot Timing Automation

As per Bug 1045938, the plan for implementing automated boot info capturing will be described below. The end goal is the eventual creation of a system to support the continued tracking of the status of boot performance. That is to say, one should be able to simply do something like

make test-perf boottest

and receive the appropriate data. Furthermore we would like to get this information into datazilla along with all the other performance numbers we track.

Plan Outline

The simplest approach we could take for accomplishing this is as follows:

The data emitted for each stage will be:

1. Capture and store pre-gecko timing to disk (something like "/data/local/.BOOTTIMES")
   • Currently setting an android property in init, and accessing it from b2g.sh to write out to disk
   • Also capturing timing just prior to starting b2g proper
   • File in JSON format to be compatible with the other tools.
2. settings.js/shell.js emit some "created" event near beginning.
   Questions:
   • Can make test-perf handle events coming from the chrome side?
   • Is make test-perf awake early enough to catch the events?
   • If not, we could send timing information in the event itself.
3. System app (in gaia) emits some "created" event
   • Do this right before iframe creation in shell.js ?
4. System App:
   • Emit moz-* events
     • Will need to implement above events for system app.
5. Vertical homescreen
   • moz-* are emitted
   • No way of actually observing these events for homescreen currently implemented.
   • Furthermore, the timing normally collected for these events are a bit different than the kind of timing I desire.

make test-perf will consume the file, and all these events to give back meaningful data. In the pre-gecko timing one of the data points will be a pair of both "uptime" and "epoch time". Using this, we can calculate the time of boot in terms of epoch time, which is more compatible with times we have access to in the higher levels of the stack. Then, for each "created" event the current time will be recorded, etc. There may need to be additional handling for getting compatible times from the moz-* events.

If make test-perf is not active early enough to catch events from shell.js, we could have it such that the system sends an event into shell when the test runner is actually ready, and tells it to emit the event (with some stored timestamp). This depends on make test-perf being able to observe the system app in the first place.

Data Visualization

Once the data itself is collected it could be presented in datazilla. What this would look like would be for each stage in the boot process to have its own line tracked over time. The Y-axis would be the cumulative time from boot, of each stage. This allows one to easily see how the duration of each section changes over time, some parts (like the kernel) would likely stay relatively static, while we may see more fluctuations in others, like gaia or gecko for example.