900
edits
Support/MetricsDashboardPRD: Difference between revisions
category -> Support Archive
(category -> Support Archive) |
|||
| (12 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
This PRD is just a draft. | This PRD is just a draft. | ||
=Overview= | |||
Rather than collecting metrics by hand and then entering them into a Google Docs spreadsheet, we should be collecting metrics on SUMO and provide a metrics dashboard to make analysis easier. The metrics we're looking to collect are [[https://wiki.mozilla.org/Support/MetricsDashboardPRD/Metrics listed here]]. | |||
=Absolute must-haves= | =Absolute must-haves= | ||
*Metrics should be able to be categorized so that they don't just all pile up on a single page | *Metrics should be able to be categorized so that they don't just all pile up on a single page | ||
*Individual metrics may have different date ranges (e.g. last week, last month, etc.) and should be clearly labeled as such | *Individual metrics may have different date ranges (e.g. last week, last month, etc.) and should be clearly labeled as such | ||
** For example, we might want to look at the traffic of the site on weekly basis, while we would want to look at CSAT trends on a monthly basis. | ** For example, we might want to look at the traffic of the site on weekly basis, while we would want to look at CSAT trends on a monthly basis. | ||
*Flexible framework where new metrics can easily be added. Ideally, there would be an admin panel where all metrics can be added/edited/removed dynamically | |||
** Properties of an individual metrics object would include title, display format (%, number, etc), category, and period length (e.g. day, week, month) | |||
*Historic storage of metrics -- similar to how we're saving metrics over time on the [http://spreadsheets.google.com/ccc?key=piA-a-dXCL2p7vB5pTu0HKA&hl=en Weekly Metrics spreadsheet] | *Historic storage of metrics -- similar to how we're saving metrics over time on the [http://spreadsheets.google.com/ccc?key=piA-a-dXCL2p7vB5pTu0HKA&hl=en Weekly Metrics spreadsheet] | ||
** The data needs to be stored in a way that allows us plot the data (piecharts, histograms) in the future. E.g. stored in the db or in xml files or similar. | ** The data needs to be stored in a way that allows us plot the data (piecharts, histograms) in the future. E.g. stored in the db or in xml files or similar. | ||
=Should be considered for inclusion= | =Should be considered for inclusion= | ||
*TikiWiki integration -- | *TikiWiki integration -- a lot of the data is coming straight from the TikiWiki db. Then there is a portion of data coming from site analytics software (in our case Omniture). Having the dashboard integrated in TikiWiki would simplify portability to other SUMO installations. | ||
*Automatic import of data from Omniture -- there is an API to pull data from Omniture, so this should be possible. Alternatively, Omniture might be able to export/send data. | *(Firefox Support specific) Automatic import of data from Omniture -- there is an API to pull data from Omniture, so this should be possible. Alternatively, Omniture might be able to export/send data. | ||
*Dynamic reporting -- i.e. specify what you want and what time period and have it generate just that report rather than only showing static reports | *Dynamic reporting -- i.e. specify what you want and what time period and have it generate just that report rather than only showing static reports | ||
*Tabbed/folded interface (because otherwise it'd be very cluttered, or the page of stats will be very long) | *Tabbed/folded interface (because otherwise it'd be very cluttered, or the page of stats will be very long) | ||
| Line 29: | Line 33: | ||
** Do we store the data we periodically gather in a way that enables us to select date ranges for the data we're collecting? | ** Do we store the data we periodically gather in a way that enables us to select date ranges for the data we're collecting? | ||
** Or are we just going for a list of SQL queries that are cron-jobbed daily/weekly/monthly and just stored statically? | ** Or are we just going for a list of SQL queries that are cron-jobbed daily/weekly/monthly and just stored statically? | ||
* | * TikiWiki integration? | ||
** | ** Having this integrated would make it easy for other installations of SUMO (e.g. Fennec, Thunderbird) to use the same features. Also, a lot of the data is coming from TikiWiki (although we probably should be using the db copy of the sumotools server for performance reasons; however that could be a pref for other SUMO installations). | ||
** Other SUMO installations might not have Omniture -- we would need an interface between website metrics and TikiWiki metrics so Omniture can easily be replaced with e.g. Urchin, Google Analytics, or whatever other sites might be using | ** Other SUMO installations might not have Omniture -- we would need an interface between website metrics and TikiWiki metrics so Omniture can easily be replaced with e.g. Urchin, Google Analytics, or whatever other sites might be using | ||
*** We could store the data we get from the Omniture API separately so that the only interfacing can be done through an import script for Urchin or Google Analytics | |||
*** Alternately, since the data may actually be different, the queries that rely on them can be dropped and replaced by appropriate queries by the individual implementation. | |||
=Storage= | |||
* We should figure out how to store this data. It doesn't seem a good idea to query the DB each time the page is generated (even if that's weekly) and have to requery to get old data. Instead, we should save the metrics in some sort of table or database structure that allows easy access. | |||
* The exact schema used is pretty important and something we should all agree on first because it's likely that it will be hard to change down the line. | |||
* Having per-article metrics that also have historical data would be a great idea but needs a good database schema to make for easy querying and data extraction. That way if we want to go back and look at top 10 articles in a given metric rather than just top 5, it's doable. This probably needs its own table(s). | |||
* We should (I should) figure out how much data we're talking about here: | |||
** We have about 12 metrics (may be more in the future) per article, about 250 articles, daily. That's 3000 numbers to store (rows?) per day if we shove everything into one table. Does this scale? Is this idea feasible? | |||
* Since none of the data refreshes more than daily, it may also be possible to shunt the current data run into a single static page that updates daily while still preserving the underlying data. | |||
=Mockup?= | |||
* Still entirely a draft but here's an initial idea (take 3): | |||
[[File:MetricsDashboardMockup.png]] | |||
* Tabs that correspond to [[Support/MetricsDashboardPRD/Metrics]]. | |||
** Groups of metrics are in tabs | |||
*** Extra [+] button for easy addition of metrics (if possible) | |||
** Clear header showing date range these metrics apply for. | |||
*** Change date range using a dropdown. Certain metrics are only collected weekly (marked with (-) in the [[Support/MetricsDashboardPRD/Metrics|metrics list]]) or weekly and monthly (marked with @) and thus can't be displayed when custom date ranges are used. | |||
** Metrics are shown in a list on each tab | |||
*** Clicking on the metric expands it to show 5 rows of historical data, as a list. Also possible: linking to a second page. | |||
*** Provide a link at the bottom of this table to get complete data. | |||
*** Provide data and a change from the previous week: green for improving, red for worsening. | |||
** List based metrics are shown as a list with appropriate sorting data. | |||
[[Category:Support Archive]] | |||