Firefox OS/Performance/Automation/Launch Latency

Launch Latency

Summary

Measures elapsed time from the first launch of an application to a series of defined startup events.

Precision

This would be filled in with the estimated precision of the test based on result variability, etc. We might estimate a precision of 10ms or we might estimate a precision of 100ms. The latter would imply very different margins before we could assume a regression.

Scenarios

Launch each of the following apps from a cold state:

• Browser
• Camera
• Contacts
• Dialer
• Email
• Gallery
• Marketplace
• Settings
• SMS

For each app, report the time elapsed to the following events:

• moz-chrome-dom-loaded
• moz-chrome-interactive
• moz-app-visually-complete
• moz-content-interative
• moz-app-loaded

Dashboards

This should be a series of links to dashboards per branch, device, and test. Unfortunately, Datazilla doesn't seem to allow direct linking without embedding revisions, specific implemented apps, and other time-specific artifacts in the URL. This is a real problem.

2.0

Test	Flame
startup_>_moz-chrome-dom-loaded	link
startup_>_moz-chrome-interactive	link
startup_>_moz-app-visually-complete	link
startup_>_moz-content-interative	link
startup_>_moz-app-loaded	link

2.1

Test	Flame
startup_>_moz-chrome-dom-loaded	link
startup_>_moz-chrome-interactive	link
startup_>_moz-app-visually-complete	link
startup_>_moz-content-interative	link
startup_>_moz-app-loaded	link

Procedure

This needs more detail.

1. Clear app from memory
2. Inject launch command
3. Monitor app events for each of the five events under Scenarios
4. Time elapsed is the time between that event and start time?

References

TBD

Roadmap

Overview

• Lead: Eli Perelman
• Tracking Bug: TBD

Milestone 1: Design

Tracking Bug: TBD

	Target Release	Test Design	Validity Review
All	TBD	Bug	Bug

Milestone 2: Test

Tracking Bug: TBD

	Target Release	Instrumentation	Workloads	Single-run test	Results Review
Browser	TBD	Bug	Bug	Bug	Bug
Camera	TBD	Bug	Bug	Bug	Bug
Contacts	TBD	Bug	Bug	Bug	Bug
Dialer	TBD	Bug	Bug	Bug	Bug
Email	TBD	Bug	Bug	Bug	Bug
Gallery	TBD	Bug	Bug	Bug	Bug
Marketplace	TBD	Bug	Bug	Bug	Bug
Settings	TBD	Bug	Bug	Bug	Bug
SMS	TBD	Bug	Bug	Bug	Bug

Milestone 3: Publication

Tracking Bug: TBD

	Target Release	Published Results	Documentation
Browser	TBD	Bug	Bug
Camera	TBD	Bug	Bug
Contacts	TBD	Bug	Bug
Dialer	TBD	Bug	Bug
Email	TBD	Bug	Bug
Gallery	TBD	Bug	Bug
Marketplace	TBD	Bug	Bug
Settings	TBD	Bug	Bug
SMS	TBD	Bug	Bug

template