Latest revision as of 23:38, 24 April 2008

This a wishlist of AMO and Addons key metrics.

Some of these metrics already exist in various parts of the AMO admin control panel but may require consolidation

STATUS

Phase I is defined and being tracked by bug 402300

TO DO

Map out the user flows and indicate what we currently measure and what we need to measure at each point in the flow (e.g. instrument button clicks, count impressions, etc...)
Draw the Venn of users and how they breakdown along different dimensions
Should we add an uninstall ping?
What happens when an add-on updates itself to version n+1 but doesn't include localization data for an existing locale? (e.g. you have pl installed and then upgrade to n+1 which doesn't have a pl localization resources)
Add a time element to ALL of the data below

Global

% of Firefox users who have "tried" an add-on (and absolute number) [May be impossible to measure]
% of Firefox users with 1 add-on installed (and absolute number)
% of Firefox users with 2 add-ons installed (and absolute number)
% of Firefox users with 3 add-ons installed (and absolute number)
% of Firefox users with 4 add-ons installed (and absolute number)
% of Firefox users with 5 add-ons installed (and absolute number)
% of Firefox users with at more than 5 add-ons installed (and absolute number)
Histogram based on number of add-ons installed

Add-ons Population

(Ap=AMO-hosted public, As=AMO-hosted sandboxed, N=Non-AMO hosted)

Total add-on population: grand total (by Ap, As, N)

Per app by add-on type: Extensions Themes, Dictionaries, Search Plugins, Plugins & Lang Packs (by Ap, As, N)

Total add-on population for Mozilla platform
- Firefox
- Thunderbird
- Seamonkey
- Sunbird
- Flock, Miro, Songbird, etc...

Competitor offerings: Safari plugins, IE add-ons, Opera

Add-ons Velocity

During the last week/month, which category areas have seen the most activity? (Top 10)

- Most addons updates & nominated from sandbox
- Most user reviews
- Most user discussion
Top 20 most highly rated addons for all time/last week/last month

For specific add-on, which ones have had the most activity. As a % of reviews/download, reviews/update ping, etc... (or other buzz metric)

Add-ons Downloads & Activity

Total number of addons downloaded all time/last week/last month
- By Browser Version
- By Operating System
- By Locale
- By geoip?
Top 20 most downloaded addons for all time/last week/last month
Top 20 most downloaded (non-recommended) addons for all time/last week/last month
Peak Session Count for last week, last month, average session count per day

Add-On Usage

Total number of active daily pings (today, avg for last week, avg for last month)
- By Locale
- By Browser Version
- By OS
Total number of active users
Average number of Addons Per User
Percent of Firefox Users with at least n add-ons
Top 100 most update ping'ed add-ons
- By Browser Version
- By OS
- By Locale
Overall distribution of specific add-ons vs. Firefox add-on user population

Basic Site Stats (urchin/Google Analytics)

Page Views
- {By Browser Version, OS}
Unique Users
Session Counts
Click paths
Singular add-on impression data (how often is X shown). Need to consider PV's as well as API
Top 50 search terms
- {By AMO Site Locale - e.g. 60% en-US, etc...}

Localization Stats

For each supported locale
- Add-On Availability/Coverage
- Add-On Metadata Availability/Coverage
- AMO Localization

Localization Metrics for langpack producers

This is not just AMO but the main l10n work in the product
Pings from active users
Downloads (separate from upgrades)
CSV format
Platform data

Addons Developers view

Downloads per week/month (preferably as CSV data) - to see new user penetration. Should differentiate upgraders from new users
Pings per week/month - to see active users.

Editor Community Metrics

How many Reviews are currently in the Moderation Queue
How many Add-Ons are currently in the Pending Updates Queue
How many Add-Ons are currently in the Nominated Queue

Efficiency measure - Over the last week/month

How many Reviews were processed from the Moderation Queue
How many Add-Ons were processed from the Pending Updates Queue
How many Add-Ons were processed from the Nominated Queue
What is the average time in the queue? Is the avg age improving over time?

How many editors do we have?
What percentage of the work over the past week/month is done by each editor?

@@ Line 5: / Line 5: @@
 = STATUS =
 * [https://intranet.mozilla.org/AMO:Dashboard-P1_AMO_Dashboard_-_Phase_I Phase I] is defined and being tracked by {{bug|402300}}
+= TO DO =
+* Map out the user flows and indicate what we currently measure and what we need to measure at each point in the flow (e.g. instrument button clicks, count impressions, etc...)
+* Draw the Venn of users and how they breakdown along different dimensions
+* Should we add an uninstall ping?
+* What happens when an add-on updates itself to version n+1 but doesn't include localization data for an existing locale? (e.g. you have pl installed and then upgrade to n+1 which doesn't have a pl localization resources)
+* Add a time element to ALL of the data below
 = Global =
-* % of Firefox users who have "tried" an add-on (and absolute number)
+* % of Firefox users who have "tried" an add-on (and absolute number) [May be impossible to measure]
 * % of Firefox users with 1 add-on installed (and absolute number)
 * % of Firefox users with 2 add-ons installed (and absolute number)
@@ Line 17: / Line 24: @@
 = Add-ons Population =
-(H=AMO-hosted public, S=sandboxed, O=offsite)
+(Ap=AMO-hosted public, As=AMO-hosted sandboxed, N=Non-AMO hosted)
-* How many grand total (hosted, sandboxed, offsite)?
+* Total add-on population: grand total (by Ap, As, N)
-* By add-on type:
+* Per app by add-on type: Extensions Themes, Dictionaries, Search Plugins, Plugins & Lang Packs  (by Ap, As, N)
-* Themes, Dictionaries, Search Plugins, Plugins, Lang Packs,
-* Firefox Themes: {by H S O}
+* Total add-on population for Mozilla platform
-* Firefox Extensions: {by H S O}
-* Total supporting each app: (H, S, O)
 ** Firefox
 ** Thunderbird
 ** Seamonkey
 ** Sunbird
-** Flock, etc...
+** Flock, Miro, Songbird, etc...
 * Competitor offerings: Safari plugins, IE add-ons, Opera
@@ Line 42: / Line 45: @@
 ** Most user discussion
 * Top 20 most highly rated addons for all time/last week/last month
+* For specific add-on, which ones have had the most activity. As a % of reviews/download, reviews/update ping, etc... (or other buzz metric)
 = Add-ons Downloads & Activity =
@@ Line 48: / Line 53: @@
 ** By Operating System
 ** By Locale
+** By geoip?
 * Top 20 most downloaded addons for all time/last week/last month
 * Top 20 most downloaded (non-recommended) addons for all time/last week/last month
@@ Line 53: / Line 59: @@
 = Add-On Usage =
-* Total number of active daily pings (today, avg for last week, avg for last month)
+* Total number of active '''daily''' pings (today, avg for last week, avg for last month)
 ** By Locale
 ** By Browser Version
@@ Line 70: / Line 76: @@
 ** {By Browser Version, OS}
 * Unique Users
-* Page Views (Search Result Pages)
+* Session Counts
+* Click paths
+* Singular add-on impression data (how often is X shown). Need to consider PV's as well as API
 * Top 50 search terms
 ** {By AMO Site Locale - e.g. 60% en-US, etc...}
@@ Line 78: / Line 86: @@
 ** Add-On Availability/Coverage
 ** Add-On Metadata Availability/Coverage
+** AMO Localization
 == Localization Metrics for langpack producers ==
@@ Line 95: / Line 104: @@
 * How many Add-Ons are currently in the Nominated Queue
-Over the last week/month
+Efficiency measure - Over the last week/month
 * How many Reviews were processed from the Moderation Queue
 * How many Add-Ons were processed from the Pending Updates Queue
 * How many Add-Ons were processed from the Nominated Queue
+* What is the average time in the queue? Is the avg age improving over time?
 * How many editors do we have?
 * What percentage of the work over the past week/month is done by each editor?

Update:RequirementsStats: Difference between revisions