Auto-tools/Projects/DevelopmentMetrics: Difference between revisions
< Auto-tools | Projects
Jump to navigation
Jump to search
Klahnakoski (talk | contribs) (add Requires VPN) |
Klahnakoski (talk | contribs) (More dashboards!) |
||
| Line 8: | Line 8: | ||
== Bug Counts == | == Bug Counts == | ||
{| | {| class="wikitable" | ||
|- | |- | ||
| style="vertical-align:top;width:300px;" | [http://people.mozilla.org/~klahnakoski/modevmetrics/Bug-Counts.html Bug Counts] || Simple count of open bugs over time, with open and close rates, and median age | | style="vertical-align:top;width:300px;" | [http://people.mozilla.org/~klahnakoski/modevmetrics/Bug-Counts.html Bug Counts] || Simple count of open bugs over time, with open and close rates, and median age | ||
| Line 19: | Line 19: | ||
==Patch Reviews== | ==Patch Reviews== | ||
{| | {| class="wikitable" | ||
|- | |- | ||
| style="vertical-align:top;width:300px;" | [http://people.mozilla.org/~klahnakoski/review/Review-byTop.html Top 30 Reviewers]<br>Requires VPN | | style="vertical-align:top;width:300px;" | [http://people.mozilla.org/~klahnakoski/review/Review-byTop.html Top 30 Reviewers]<br>Requires VPN | ||
| Line 35: | Line 35: | ||
| style="vertical-align:top;width:300px;" | [http://people.mozilla.org/~klahnakoski/review/Reviews_Pending_18.html Reviews Requested in the Past 18 weeks]<br>Requires VPN | | style="vertical-align:top;width:300px;" | [http://people.mozilla.org/~klahnakoski/review/Reviews_Pending_18.html Reviews Requested in the Past 18 weeks]<br>Requires VPN | ||
| An attempt to better understand the aggregate response time; while treating "old" reviews (those over 18weeks), as something else entirely. | | An attempt to better understand the aggregate response time; while treating "old" reviews (those over 18weeks), as something else entirely. | ||
|} | |||
==Release Management Dashboards== | |||
[https://wiki.mozilla.org/Release_Management Release Management] are responsible for quality, security, and stability. | |||
{| class="wikitable" | |||
|- | |||
| style="vertical-align:top;width:300px;" | [http://people.mozilla.org/~klahnakoski/platform/releases.html Release Dashboard] | |||
| All outstanding bugs tracked by the Release Management | |||
|- | |||
| style="vertical-align:top;width:300px;" | [http://people.mozilla.org/~klahnakoski/platform-history/release-history.html Release Uplifts] | |||
| Count the number of patches that have been uplifted to the Aurora and Beta branches. | |||
|} | |||
== Contributors == | |||
{| class="wikitable" | |||
|- | |||
| style="vertical-align:top;width:300px;" | [http://people.mozilla.org/~klahnakoski/contributors/dashboard.html# Contributors Dashboard] | |||
| [https://wiki.mozilla.org/EngineeringProductivity Engineering Productivity's] list of contributor bugs that may benefit from some action on our part. | |||
|} | |} | ||
| Line 41: | Line 62: | ||
These dashboards are for showing general project state and burndown. | These dashboards are for showing general project state and burndown. | ||
{| | {| class="wikitable" | ||
|- | |- | ||
| style="vertical-align:top;width:300px;" | [http://people.mozilla.org/~klahnakoski/modevmetrics/Dashboard_byProject.html Project Dashboard] | | style="vertical-align:top;width:300px;" | [http://people.mozilla.org/~klahnakoski/modevmetrics/Dashboard_byProject.html Project Dashboard] | ||
| Line 58: | Line 79: | ||
In the past there have been some for specific historical charts, and simple trend analysis.<br> | In the past there have been some for specific historical charts, and simple trend analysis.<br> | ||
{| | {| class="wikitable" | ||
|- | |- | ||
| style="vertical-align:top;width:300px;" | [http://people.mozilla.org/~klahnakoski/modevmetrics/B2G-TriageRate.html FFOS Bug Triage (Nominations)] | | style="vertical-align:top;width:300px;" | [http://people.mozilla.org/~klahnakoski/modevmetrics/B2G-TriageRate.html FFOS Bug Triage (Nominations)] | ||
Revision as of 19:59, 19 January 2016
Overview
These charts serve two purposes. The first is to satisfy one of the specific needs of the Bugzilla Anthropology Project which was to provide a view into patch review times. The second is to prove ElasticSearch can be used a performant data warehouse. More on the project and conclusions can be found at the Jan 2013 project summary. There is also my DashCon presentation( PPT)
Bug Counts
| Bug Counts | Simple count of open bugs over time, with open and close rates, and median age
chart. |
| Bug Close Rate | Much better array of filters available |
Patch Reviews
| Top 30 Reviewers Requires VPN |
Shows the top reviewers from the past 6 weeks, the number of reviews they did, and the number in their queue right now. The original objective of this dashboard is to confirm some engineers have too many reviews. This dashboard is fine for identifying egregious differences, but poor for individual comparison; it seems personal style, team style, and code under review all impact the size and number of reviews done. |
| Time to First Incoming Review Requires VPN |
Counts review requests and review completion rates. Also breaks down aggregate review age. The first review has been shown to take the longest, probably due to people synchronization issues. Subsequent reviews were almost instantaneous, so we exclude them. This dashboard is not good for fine grained comparison of teams because the nature of the bugs, and the process used, is often too different. The response time INCLUDES WEEKENDS. |
| Incoming Review by Individual Requires VPN |
Top reviewers, but this time with some breakdown by week and component. Interesting, but not actionable. |
| Incoming Reviews Requires VPN |
Statistics for all reviews. Subsequent reviews of a bug happen much faster than the first. Maybe the fast review speed is an administrative anomaly; reviews are added after the fact, and marked as reviewed. Maybe, the requester and reviewer are working closely to remove the remaining nits. |
| Reviews Requested in the Past 18 weeks Requires VPN |
An attempt to better understand the aggregate response time; while treating "old" reviews (those over 18weeks), as something else entirely. |
Release Management Dashboards
Release Management are responsible for quality, security, and stability.
| Release Dashboard | All outstanding bugs tracked by the Release Management |
| Release Uplifts | Count the number of patches that have been uplifted to the Aurora and Beta branches. |
Contributors
| Contributors Dashboard | Engineering Productivity's list of contributor bugs that may benefit from some action on our part. |
Project Dashboards and Burndown
These dashboards are for showing general project state and burndown.
| Project Dashboard | Lists assigned bugs, patches in review, and patches awaiting landing |
| Burndown Prediction | Uses average new bug rate, and average close rate to estimate the time it takes
to close all bugs in the given category. Compares to close rates necessary to meet due date |
| Final Burndown Dashboard | Basic burndown chart. Showing current bugs over time compared to ideal (linear) burndown |
Specific Projects
In the past there have been some for specific historical charts, and simple trend analysis.
| FFOS Bug Triage (Nominations) | The nomination queue can be filtered by product and component. It depends on Mozilla's standard practice using a tracking flag to mark bug one of "?" (triage), "+" (tracked), "-" (not tracked) |
| TEF vs B2G Fix Rate | Compare the fix rates for Boot2Gecko product and TEF+ blockers |
| B2G Close Rate | Derived from Bug Close Rate to show four major B2G projects, and only the Resolved=FIXED. |
| Security's 2013 Q1 Goal | The Security Teams have an ambitious goal to limit the age of high+ security bugs. This dashboard is meant to help visualize progress toward those goals. |
| Security_byPriority | Count security bugs by priority, over time |
| Security_byRisk.html | Show risk assessment by component, over time. It uses security's scoring method to give a number to each of the components |