Confirmed users, Bureaucrats and Sysops emeriti
1,217
edits
Telemetry/Experiments: Difference between revisions
Jump to navigation
Jump to search
Clarify some things based on lmandel's feedback.
(trivial) |
(Clarify some things based on lmandel's feedback.) |
||
| Line 11: | Line 11: | ||
== Data Collection == | == Data Collection == | ||
Before building an experiment, contact [https://mozillians.org/en-US/u/bsmedberg/ bsmedberg] to talk about experiment design and how the results of the experiment will be measured and reported. | Before building an experiment, contact [https://mozillians.org/en-US/u/bsmedberg/ bsmedberg] or a data-collection peer to talk about experiment design and how the results of the experiment will be measured and reported. Depending on the nature of the experiment and its data-collection system, he may require additional privacy or security review. | ||
== Engineering == | == Engineering == | ||
Experiments must be developed in a public source repository and must be reviewed just as if they were part of Firefox on mozilla-central | Experiments must be developed in a public source repository and must be reviewed just as if they were part of Firefox on mozilla-central. | ||
Experiments are restartless .xpi and can do anything a normal Firefox extension can do. In install.rdf, em:type==128 must be used. | Experiments are restartless .xpi and can do anything a normal Firefox extension can do. In install.rdf, em:type==128 must be used. | ||
| Line 27: | Line 27: | ||
== Final Deployment == | == Final Deployment == | ||
Final deployment of an experiment must be approved by release- | Final deployment of an experiment must be approved by [https://mozillians.org/en-US/u/lmandel/ Lawrence Mandel] or a release-management peer. Send an email to release-drivers@mozilla.org with the following information: | ||
* A brief summary of the experiment, the questions that will be answered, and how the experiment will answer the questions | * A brief summary of the experiment, the questions that will be answered, and how the experiment will answer the questions | ||
* bug(s) where the code was developed and reviewed | * bug(s) where the code was developed and reviewed | ||
* details about experiment deployment: what channel(s) will the experiment be deployed to, how many users will be participating, and how long will the experiment run? This data can be copied from the [https://telemetry-experiment-dev.allizom.org/ staging server]. | * details about experiment deployment: what channel(s) will the experiment be deployed to, how many users will be participating, and how long will the experiment run? This data can be copied from the [https://telemetry-experiment-dev.allizom.org/ staging server]. | ||
Here is an example of an approval email: | |||
<pre> | |||
Subject: Intent to ship: Telemetry experiment for verification of telemetry experiments bug | |||
This is a request for approval to ship the following Telemetry experiment: | |||
Telemetry experiment for verification of telemetry experiments bug. | |||
Summary | |||
In bug 1052714, I plan to ship a telemetry experiment to 50% of nightly channel users to verify that a fix for a race condition setting telemetry experiment branches is actually fixed. This experiment will have no user effect, and only exists to test the system. The results will be collected and analyzed from FHR payloads. | |||
Related Bugs: | |||
bug 1052714 | |||
Start Date | |||
14-Aug-2014 | |||
End Date | |||
10-Sep-2014 | |||
Maximum runtime | |||
7.0 days | |||
Update Channels | |||
nightly | |||
Build ID | |||
20140820030202- | |||
Instance population | |||
All nightly instances | |||
Sample Rate | |||
50%</pre> | |||
After final approval, file a bug to deploy to production at https://telemetry-experiment.cdn.mozilla.net/ . Use {{bug|1057464}} as a model for deployment. | After final approval, file a bug to deploy to production at https://telemetry-experiment.cdn.mozilla.net/ . Use {{bug|1057464}} as a model for deployment. | ||