Firefox OS/Performance/Boot Timing Automation

As per Bug 1045938, the plan for implementing automated boot info capturing will be described below. The end goal is the eventual creation of a system to support the continued tracking of the status of boot performance. That is to say, one should be able to simply do something like

make test-perf boottest

and receive the appropriate data. Furthermore we would like to get this information into datazilla along with all the other performance numbers we track.

Plan Outline

This primarily relates to Bug 1047118. The simplest approach we could take for accomplishing this is as follows:

The data emitted for each stage will be:

1. Capture and store pre-gecko timing to disk (something like "/data/local/.BOOTTIMES")
   • Currently setting an android property in init, and accessing it from b2g.sh to write out to disk
   • Also capturing the timing of when we launch b2g proper
   • File in JSON format to be compatible with the other tools.
2. settings.js/shell.js generate a time stamp (that is stored in some standard property of the window) near its entrypoint.
   
3. System app (in chrome) records timing at some decided location (as in the prior step):
   • Do this right before iframe creation in shell.js ?
4. System App:
   • Record time stamp at entry point, like previously.
5. Vertical homescreen
   • Record time stamp at entry point
   • moz-* are emitted
   • There is no way of actually observing these events for homescreen currently implemented, and there needs to be work done to make use of these events being fired in my framework of things

make test-perf will consume the .BOOTTIMES file, and all the properties to give back meaningful data. In the pre-gecko timing one of the data points will be a pair of both "uptime" and "epoch time". Using this, we can calculate the time of boot in terms of epoch time, which is more compatible with times we have access to in the higher levels of the stack. Then, for each "created" event that fires, the current time will be recorded. There may need to be additional handling added for getting compatible times from the moz-* events.

In the current design of make test-perf startup_event_tests, the test designates code to be injected, and launches the app, thus having full control over what it does. This is not quite suitable for the system and verticalhome apps since they are kind of a special case. Upon gecko finishing its initialization steps, shell.html is unconditionally run, which loads the system app directly (which then also goes and prepares the lockscreen and the homescreen). The approach of injecting certain handlers may be possible however, if one reloads b2g to give the code a chance to be injected into the relevant frames. But instead of firing and catching events, we could (conditionally, upon a setting even) record time stamps in some standard object for the test harness to collect afterwards. This way there is minimal impact on the apps themselves.

When launching an app, we get to change to its context, but luckily there are other ways to achieve the same effect. First of all, the top level marionette client's chrome is shell, and its content is the System app. From there we can then just find the iframe beneath that contains the homescreen app, switch to that frame, and collect the last bit of data.

Data Visualization

Once the data itself is collected it could be presented in datazilla. What this would look like would be for there to be a Boot test, and for each stage in the boot process to have its own line tracked over time. The Y-axis would be the cumulative time from boot, of each stage. This allows one to easily see how the duration of each section changes over time. Some parts (like the kernel) would likely stay relatively static, while we may see more fluctuations in others (gaia or gecko for example).