Confirmed users
14,525
edits
QA/Platform/DOM/Metrics: Difference between revisions
Jump to navigation
Jump to search
→Pending Investigation
| (13 intermediate revisions by the same user not shown) | |||
| Line 12: | Line 12: | ||
In implementation we will experiment with a lot of different metrics and data sources in an attempt to see which metrics, either singularly or in conjunction with other data, enable us to effectively make informed decisions. | In implementation we will experiment with a lot of different metrics and data sources in an attempt to see which metrics, either singularly or in conjunction with other data, enable us to effectively make informed decisions. | ||
=== | === Drafted === | ||
The following metrics were developed using data from [http://people.mozilla.org/~klahnakoski/modevmetrics/ Kyle Lahnakoski's] elastic search Bugzilla database. Pending peer review, the next step will be to evaluate [https://github.com/mozilla/metrics-graphics better charting] and eventually a more automated solution. | |||
* [https://docs.google.com/a/mozilla.com/spreadsheets/d/1i9jGwCZDOXJlSGrmIgvyHzBQbTG5aPEi6Np_P-4GDwk/edit?usp=sharing Unresolved crash bugs by component] - ''possibly correlate to KairRo's reports:'' | |||
** https://crash-analysis.mozilla.com/rkaiser/2015-02-05/2015-02-05.firefox.release.35.0.components.html | |||
** https://crash-analysis.mozilla.com/rkaiser/2015-02-05/2015-02-05.firefox.beta.36.0.components.html | |||
** https://crash-analysis.mozilla.com/rkaiser/2015-02-05/2015-02-05.firefox.aurora.components.html | |||
** https://crash-analysis.mozilla.com/rkaiser/2015-02-05/2015-02-05.firefox.nightly.components.html | |||
* <strike>[https://docs.google.com/a/mozilla.com/spreadsheets/d/1epRHLg58XmW90-SwCzFmmhqy-_SAeoh76_F9WXi-NQc/edit?usp=sharing Unresolved intermittent failure bugs by component]</strike> ''Deprioritized for further investigation following discussion with Andrew Overholt'' | |||
* [https://docs.google.com/a/mozilla.com/spreadsheets/d/1lJSH_pnnj0k-fYP_W4jZ9GYuEe7DW32YwuL1nw61rd8/edit?usp=sharing Unresolved regression bugs by component] | |||
* [https://docs.google.com/a/mozilla.com/spreadsheets/d/18JHF8otni_5bJHFVqt6ImsVuX5MQrSLW23fXX0QOAQc/edit?usp=sharing Unconfirmed bugs by component] | |||
* [https://github.com/ashughes1/mozbackouts Backouts by component] | |||
* [https://docs.google.com/a/mozilla.com/spreadsheets/d/1pafr15_q0ztifEMY-bHEQWlQBkPrmq7e3ebU8JS_vqg/edit#gid=1448481380 Regressions resolved prior to Beta] (updated as of Firefox 37) | |||
=== Pending Investigation === | |||
''Draft proof of concepts can be found in my [https://drive.google.com/drive/u/1/#folders/0B9_ZEZ4TIS-8M2ZaTDFXT20wMDA Google Drive]'' | ''Draft proof of concepts can be found in my [https://drive.google.com/drive/u/1/#folders/0B9_ZEZ4TIS-8M2ZaTDFXT20wMDA Google Drive]'' | ||
* https://github.com/AutomatedTester/futurama-data | * https://github.com/AutomatedTester/futurama-data | ||
** backout rate | ** backout rate | ||
* http://brasstacks.mozilla.com/testreports/weekly | * http://brasstacks.mozilla.com/testreports/weekly | ||
| Line 43: | Line 55: | ||
** https://scitools.com/ for code complexity analysis, possibly other metrics | ** https://scitools.com/ for code complexity analysis, possibly other metrics | ||
** http://metricsgraphicsjs.org/ for JS library to create dashboards | ** http://metricsgraphicsjs.org/ for JS library to create dashboards | ||
** Telemetry | |||
** FHR | |||
** Socorro | |||
** [http://release.mozilla.org/statistics/37/2015/03/13/fx-37-b4-to-b5.html Release Management] | |||
* [http://brasstacks.mozilla.com/orangefactor/ OrangeFactor] | |||
== Bugzilla Data == | |||
=== Using Manual Querying === | |||
=== Using Elastic Search === | |||