Auto-tools/Projects/DevelopmentMetrics: Difference between revisions
< Auto-tools | Projects
Jump to navigation
Jump to search
Klahnakoski (talk | contribs) (add placeholder) |
Klahnakoski (talk | contribs) (first draft) |
||
| Line 1: | Line 1: | ||
== Overview == | |||
These charts serve two purposes. The first is to satisfy one of the specific needs of the [[Bugzilla Anthropology|Bugzilla Anthropology Project]] which was to provide a view into patch review times. The second is to prove | |||
ElasticSearch can be used a performant [http://en.wikipedia.org/wiki/Data_warehouse|data warehouse]. | |||
More on the project and conclusions can be found at the [https://wiki.mozilla.org/index.php?title=Bugzilla_Anthropology/2013-01-29|Jan 2013 project summary]. | |||
There is also my [http://people.mozilla.org/~klahnakoski/docs/Bugs%20in%20ES.pdf|DashCon presentation]( [http://people.mozilla.org/~klahnakoski/docs/Bugs%20in%20ES.pptx|PPT]) | |||
== Bug Counts == | |||
{| | |||
|- | |||
| [http://people.mozilla.org/~klahnakoski/modevmetrics/Bug-Counts.html Bug Counts] || Simple count of open bugs over time, with open and close rates, and median age | |||
chart. | |||
|- | |||
| [http://people.mozilla.org/~klahnakoski/modevmetrics/Bug-CloseRate.html Bug Close Rate] || Much better array of filters available | |||
|} | |||
==Project Dashboards and Burndown== | |||
These dashboards are for showing general project state and burndown. | |||
{| | |||
|- | |||
| [http://people.mozilla.org/~klahnakoski/modevmetrics/Dashboard_byProject.html Project Dashboard] || Lists assigned bugs, patches in review, and patches awaiting landing | |||
|- | |||
| [http://people.mozilla.org/~klahnakoski/modevmetrics/Predictive_Simple.html#programFilter=B2G+1.3 Burndown Prediction] || Uses average new bug rate, and average close rate to estimate the time it takes | |||
to close all bugs in the given category. Compares to close rates necessary to meet due date | |||
|- | |||
| [http://people.mozilla.org/~klahnakoski/modevmetrics/Dashboard-FinalBurndown.html Final Burndown Dashboard] || Basic burndown chart. Showing current bugs over time compared to ideal (linear) burndown | |||
|} | |||
==Specific Projects== | |||
In the past there have been some for specific historical charts, and simple trend analysis.<br> | |||
{| | |||
|- | |||
| [http://people.mozilla.org/~klahnakoski/modevmetrics/B2G-TriageRate.html FFOS Bug Triage (Nominations)] || The nomination queue can be filtered by product and component. It depends on Mozilla's standard practice using a tracking flag to mark bug one of "?" (triage), "+" (tracked), "-" (not tracked) | |||
|- | |||
| [http://people.mozilla.org/~klahnakoski/modevmetrics/TEF-CloseRate.html TEF vs B2G Fix Rate] || Compare the fix rates for Boot2Gecko product and TEF+ blockers | |||
|- | |||
| [http://people.mozilla.org/~klahnakoski/modevmetrics/B2G-CloseRate.html B2G Close Rate] || Derived from Bug Close Rate to show four major B2G projects, and only the Resolved=FIXED. | |||
|- | |||
| [http://people.mozilla.org/~klahnakoski/modevmetrics/Security_Q1_Goal.html Security's 2013 Q1 Goal] || The Security Teams have an ambitious goal to limit the age of high+ security bugs. This dashboard is meant to help visualize progress toward those goals. | |||
|- | |||
| [http://people.mozilla.org/~klahnakoski/modevmetrics/Security_byPriority.html Security_byPriority] || Count security bugs by priority, over time | |||
|- | |||
| [http://people.mozilla.org/~klahnakoski/modevmetrics/Security_byRisk.html Security_byRisk.html] || Show risk assessment by component, over time. It uses security's scoring method to give a number to each of the components | |||
|} | |||
==Patch Reviews== | |||
{| | |||
|- | |||
| [http://people.mozilla.org/~klahnakoski/review/Review-byTop.html Top Reviewers] || Shows the top 30 reviewers from the past 6 weeks, the number of reviews they did, and the number in their queue right now | |||
|- | |||
| [http://people.mozilla.org/~klahnakoski/review/Review-byReviewer.html Reviews by Reviewer] || Top reviewers, but this time with some breakdown by week and component | |||
|- | |||
| [http://people.mozilla.org/~klahnakoski/review/ReviewIntensity_First.html ReviewIntensity_First] || Counts review requests and review completion rates. Also breaks down aggregate review age. The first review has been shown to take significantly longer | |||
|- | |||
| [http://people.mozilla.org/~klahnakoski/review/ReviewIntensity.html ReviewIntensity] || Statistics for all reviews. Subsequent reviews of a bug happen much faster than the first. It is assumed once the bug has the attention of the reviewer, the review cycle happens quickly | |||
|} | |||
Revision as of 22:50, 23 June 2015
Overview
These charts serve two purposes. The first is to satisfy one of the specific needs of the Bugzilla Anthropology Project which was to provide a view into patch review times. The second is to prove ElasticSearch can be used a performant warehouse. More on the project and conclusions can be found at the 2013 project summary. There is also my presentation( [1])
Bug Counts
| Bug Counts | Simple count of open bugs over time, with open and close rates, and median age
chart. |
| Bug Close Rate | Much better array of filters available |
Project Dashboards and Burndown
These dashboards are for showing general project state and burndown.
| Project Dashboard | Lists assigned bugs, patches in review, and patches awaiting landing |
| Burndown Prediction | Uses average new bug rate, and average close rate to estimate the time it takes
to close all bugs in the given category. Compares to close rates necessary to meet due date |
| Final Burndown Dashboard | Basic burndown chart. Showing current bugs over time compared to ideal (linear) burndown |
Specific Projects
In the past there have been some for specific historical charts, and simple trend analysis.
| FFOS Bug Triage (Nominations) | The nomination queue can be filtered by product and component. It depends on Mozilla's standard practice using a tracking flag to mark bug one of "?" (triage), "+" (tracked), "-" (not tracked) |
| TEF vs B2G Fix Rate | Compare the fix rates for Boot2Gecko product and TEF+ blockers |
| B2G Close Rate | Derived from Bug Close Rate to show four major B2G projects, and only the Resolved=FIXED. |
| Security's 2013 Q1 Goal | The Security Teams have an ambitious goal to limit the age of high+ security bugs. This dashboard is meant to help visualize progress toward those goals. |
| Security_byPriority | Count security bugs by priority, over time |
| Security_byRisk.html | Show risk assessment by component, over time. It uses security's scoring method to give a number to each of the components |
Patch Reviews
| Top Reviewers | Shows the top 30 reviewers from the past 6 weeks, the number of reviews they did, and the number in their queue right now |
| Reviews by Reviewer | Top reviewers, but this time with some breakdown by week and component |
| ReviewIntensity_First | Counts review requests and review completion rates. Also breaks down aggregate review age. The first review has been shown to take significantly longer |
| ReviewIntensity | Statistics for all reviews. Subsequent reviews of a bug happen much faster than the first. It is assumed once the bug has the attention of the reviewer, the review cycle happens quickly |