Auto-tools/Projects/DevelopmentMetrics: Difference between revisions

Revision as of 22:20, 10 December 2016

Overview

These charts serve two purposes. The first is to satisfy one of the specific needs of the Bugzilla Anthropology Project which was to provide a view into patch review times. The second is to prove ElasticSearch can be used a performant data warehouse. More on the project and conclusions can be found at the Jan 2013 project summary. There is also my DashCon presentation( PPT)

Bug Counts

Bug Counts	Open bug count over time, with open/close rates, and median age. Filter bugs by time range, components, and common program markups. This dashboard is meant as a rough-first-step to building something more process specific.
Bug Close Rate	Counts bug closures, and requires writing Elasticsearch filters to limit view to desired category of bugs.

Patch Reviews

Top 30 Reviewers Requires VPN	Shows the top reviewers from the past 6 weeks, the number of reviews they did, and the number in their queue right now. The original objective of this dashboard is to confirm some engineers have too many reviews. This dashboard is fine for identifying egregious differences, but poor for individual comparison; it seems personal style, team style, and code under review all impact the size and number of reviews done.
Time to First Incoming Review	Counts review requests and review completion rates. Also breaks down aggregate review age. The first review has been shown to take the longest, probably due to people synchronization issues. Subsequent reviews were almost instantaneous, so we exclude them. This dashboard is not good for fine grained comparison of teams because the nature of the bugs, and the process used, is often too different. The response time INCLUDES WEEKENDS.
Incoming Review by Individual Requires VPN	Top reviewers, but this time with some breakdown by week and component. Interesting, but not actionable.
Incoming Reviews	Statistics for all reviews. Subsequent reviews of a bug happen much faster than the first. Maybe the fast review speed is an administrative anomaly; reviews are added after the fact, and marked as reviewed. Maybe, the requester and reviewer are working closely to remove the remaining nits.
Reviews Requested in the Past 18 weeks	An attempt to better understand the aggregate response time; while treating "old" reviews (those over 18weeks), as something else entirely.

Engineering Dashboards

Dashboards for use by developers, or our own team

Neglected Oranges	List of current, intermittent test failures, and have een no bug activity in the past week.

Release Management Dashboards

Release Management are responsible for quality, security, and stability.

Release Dashboard	All outstanding bugs tracked by the Release Management
Release Uplifts	Count the number of patches that have been uplifted to the Aurora and Beta branches.

Contributors

Contributors Dashboard	Engineering Productivity's list of contributor bugs that may benefit from some action on our part.

Project Dashboards and Burndown

These dashboards are for showing general project state and burndown.

Project Dashboard Requires VPN	Lists assigned bugs, patches in review, and patches awaiting landing
Burndown Prediction Requires VPN	Uses average new bug rate, and average close rate to estimate the time it takes to close all bugs in the given category. Compares to close rates necessary to meet due date
Final Burndown Dashboard Requires VPN	Basic burndown chart. Showing current bugs over time compared to ideal (linear) burndown

@@ Line 22: / Line 22: @@
 | Shows the top reviewers from the past 6 weeks, the number of reviews they did, and the number in their queue right now.  The original objective of this dashboard is to confirm some engineers have too many reviews. This dashboard is fine for identifying egregious differences, but poor for individual comparison; it seems personal style, team style, and code under review all impact the size and number of reviews done.
 |-
-| style="vertical-align:top;width:300px;" | [http://people.mozilla.org/~klahnakoski/review/ReviewIntensity_First.html Time to First Incoming Review]<br><span style="color:red">Requires VPN</span>
+| style="vertical-align:top;width:300px;" | [http://people.mozilla.org/~klahnakoski/review/ReviewIntensity_First.html Time to First Incoming Review]
 | Counts review requests and review completion rates.  Also breaks down aggregate review age.  The first review has been shown to take the longest, probably due to people synchronization issues.  Subsequent reviews were almost instantaneous, so we exclude them.  This dashboard is not good for fine grained comparison of teams because the nature of the bugs, and the process used, is often too different.  The response time INCLUDES WEEKENDS.
 |-
@@ Line 28: / Line 28: @@
 | Top reviewers, but this time with some breakdown by week and component.  Interesting, but not actionable.
 |- style="vertical-align:top;"
-| [http://people.mozilla.org/~klahnakoski/review/ReviewIntensity.html Incoming Reviews]<br><span style="color:red">Requires VPN</span>
+| [http://people.mozilla.org/~klahnakoski/review/ReviewIntensity.html Incoming Reviews]
 | Statistics for all reviews.  Subsequent reviews of a bug happen much faster than the first.  Maybe the fast review speed is an administrative anomaly; reviews are added after the fact, and marked as reviewed.  Maybe, the requester and reviewer are working closely to remove the remaining nits.
 |-
-| style="vertical-align:top;width:300px;" | [http://people.mozilla.org/~klahnakoski/review/Reviews_Pending_18.html Reviews Requested in the Past 18 weeks]<br><span style="color:red">Requires VPN</span>
+| style="vertical-align:top;width:300px;" | [http://people.mozilla.org/~klahnakoski/review/Reviews_Pending_18.html Reviews Requested in the Past 18 weeks]
 | An attempt to better understand the aggregate response time; while treating "old" reviews (those over 18weeks), as something else entirely.
 |}