Firefox OS/Metrics

Goals

The FxOS Metrics project enables making data driven decisions for the following:

• Understanding how FxOS is doing in the market versus product/business goals
• Gaining insights from quantitative and qualitative data, to enable improve the product/business goals
• Understanding usage models, to improve future product development and opportunities
• Determining effectiveness of marketing campaigns.

Metrics

Customer lifecycle	Metrics to be collected	Dashboard
Acquiring Customers	Partners/Channels though which devices are sold	Dashboard: Choose Operator/OEM/Country combination of channel
Activation of Devices	Total number of Activated devices	Dashboard: Choose 'All' in SW/Operator/OEM/ to get a count
Retention Of Customers	MAU (Monthly Active Users) Apps usage per month Performance (App Startup time, Power consumption)	Internal
Revenue	Number of searches done Paid App Downloads	Internal
Referral	Net Promoter Score Recommendations from Users User Satisfaction Factor	Future

Roadmap

FxOS Release	Metrics to be collected	Bugs
1.3	Acquisition, Activations: Activation Ping:	Engg: 986701,997960 Legal: 980920 Privacy: 992487
2.1	Retention App Usage (Default Off)	Client: 982663 Legal: 1021259, 1109429
2.2	Retention App Usage (Default On) AppUsage, MAU Acquisition: Report Preinstalled Apps Revenue Num of Searches/Revenue	AppUsage, MAU: 1109422 Client side work for enabling AppUsage , 1119009 Marketplace Apps Only 1109426 Server side dashboard for Most used apps: 1021173 Report Preinstalled Apps, 1126524 Search
2.5	Retention: OTA collection of App Startup time (1179004). Activation Accuracy of activation: FTU Ping: Avoid counting repeat pings from a known device. (1171685). Privacy: Converged UX for opt-in, opt-out of metrics. 1181295. Scaling: Framework (Metabug: 1152000) to collect custom Metrics and histogram Non Phone devices support: Using TV serial number to avoid counting repeat pings. 1171685. B2GDroid Support (B2GDroid Metric Collection 1198338) AddOns Support (AddOns Metric collection 1198346)
2.6 (May 2016)	(In Planning) Meta Bug for 2.6 (1226182) Activation Optimizing FTU: % of signups to Moz services: FxAccounts, FMD Accuracy of Retention/MAU: FTU ping should include info on whether User opt-in for metrics (1198164) Retention Inclusion of device metrics in A/B testing (likely to be moved beyond 2.5) Engagement with device: Number of voice calls, data (wifi and cellular), number of text messages. 1193225. Device Startup time: OTA collection of device dtartup time (1177226) Advanced A/B testing Churn Rate & User Lifecycle Analysis (In planning) Framework Improvements Enum Type and Boolean Type histograms Support for New Security model 1225960 FoTA Ping Engg bug: 1202970, Legal bug: 1202987. Referral Net Promoter Score (User Satisfaction Factor) Privacy User ability to delete collected info. 1126513. Server Graphing of Addon Histograms Graphing of Keyed Histograms
End-2016 (In Planning)	Retention Content Recommendation Engine based on usage (User Benefit)

User Stories

• User Stories

Virtual Team

• Device Engg: Tamara Hills, Gregor Wagner, Russ Nicoletti
• Device QA: Shing Lyu
• Server Metrics: Dave Zeber, John Jensen
• Planning: Dominik Strohmeier
• Product Mgmt: Ravi Dandu
• Marketplace Product Mgmt: David Bialer, Sakina Groth
• Alumni: Marshall Culpepper, Hema Koka

Links

• CustomMetrics
• EngineeringMetrics
• Activation Ping
• AppUsage Data
• MAU Doc
• Main Metrics Page (this page)
• Testing Guide
• Frequently Asked Questions
• Performance Goals