Firefox/2015 Measurement Program

Overall Goal

Build out our data reporting infrastructure for both product metrics and engineering/quality metrics. Complete the most important reports first.

Other Goals

• Prove that the unified telemetry system and pipeline can answer questions quickly
• Build a library of analysis scripts and tools for solving problems within the new pipeline
• Build out the tools/systems necessary for other teams to build their own reporting in a self-service way

People

• Product sponsor: bsmedberg
• Engineering program manager: thuelbert
• Engineering lead: kparlante or trink or mreid?
• Firefox product management: chadw or rweiss?
• Firefox quality representative: lmandel

Strategy

• Collect and prioritize a list of use cases (from Lawrence and Firefox product management)
• For the most important report on the list:
  • Determine what reporting will solve the use case
  • Determine whether we're already collecting the necessary data
  • Collect the necessary data
  • Define the output format/aggregate structure
  • Have the data pipeline or other source output the correct data
  • Turn that into the final output (dashboard, email alert, whatever)
• Repeat for the next thing on the list

Aspects of a Report

• Who is responsible for monitoring this report/data?
• How do they want to consume the data?
• How will we know if the data continues to be monitored and useful?
• Who else may have access to this report/data?