Bugzilla Anthropology: Difference between revisions

Latest revision as of 23:55, 19 January 2014

General Agenda

To figure out how people are currently engaging with Bugzilla and see what we can learn about the tool's good, bad, and frustrating qualities.

Identify the information we want to gather.
Interview stakeholders across the organization.
Surface statistics that are relevant to it's use.
Identify best practices, useful hacks, and key problem areas.
Provide an overview of how Bugzilla is being used.

Status - Updated October 2014

The BMO/Auto-Tools team has accepted making the ElasticSearch cluster public see goal details

Status - Updated January 2013

Metrics' ETL has proven good for generating charts
See full update

Status - Updated June 4th 2012

22 interviews completed
- Current estimated required 20 to get high coverage
- Final Round of interviews, 100% complete.
- Interview being summaries by Olga Baysal and Reid Holmes of Cheriton School of Computer Science at the University of Waterloo

Progress on statistics side remains slow
- A prototype is imminent, 3 bugs continue to block this progress although some progress has been made in recent days.
- Once the database schema has been updated with the corrections, automated tests will need to be written to insure that data is trustworthy
- A understanding of what security concerns there are remaining has been worked out with the security team and there is now a clear road map of what needs to happen to make the information public.
  - Due to resource constraints, it's hard to get an ETA on when this work will be done.
- Access to current experiments can be found at the following LDAP protected URL
  - Bugzilla Dashboard Prototypes

Initial presentation: File:Bugzilla Anthropology Presentation.pdf

Interviews

Interviews will be a large portion of the programs scope. The the following section includes details about what will be investigated, who will be interviewed, and the notes gathered in those interviews.

Bugzilla Interview Questions Etherpad

Transcripts

Below are the results of the interviews conducted so far. These are the raw transcripts so they are unfiltered and the raw input that was given by the participants. Summaries of this information will be coming but these will be useful for those that want to dig a little deeper.

Summaries

‎3 Page Summary on Interview Topic Categorization: File:BA - 3 page summary.pdf

Interviews

Write Ins

Statistics

Gathering statistics on Bugzilla usage is a good way to provide an additional perspective that can help corroborate or question the information we gather via the interviews. A tool has emerged that seems to provide some solid insight into this area.

As this data significant influence on the interpretation of interview findings, verifying that the data presented is accurate is very important. This work is currently being discussed on the following Etherpad:

QA Dashboard Vetting Etherpad

@@ Line 9: / Line 9: @@
 *Provide an overview of how Bugzilla is being used.
-== Status - Updated 11/29/2011 ==
-* Interviews 10 completed
+== Status - Updated October 2014 ==
+* The BMO/Auto-Tools team has accepted making the ElasticSearch cluster public [https://wiki.mozilla.org/Auto-tools/Projects/PublicES see goal details]
+== Status - Updated January 2013 ==
+* Metrics' ETL has proven good for generating charts
+* [https://wiki.mozilla.org/index.php?title=Bugzilla_Anthropology/2013-01-29 See full update]
+== Status - Updated June 4th 2012 ==
+* 22 interviews completed
 ** Current estimated required 20 to get high coverage
-** Transcripts posted below
+** Final Round of interviews, 100% complete.
-** Next ten interviews will be determined once initial batch have been reviewed
+** Interview being summaries by Olga Baysal and Reid Holmes of Cheriton School of Computer Science at the University of Waterloo
-* Currently working to get visibility on how review queue size is changing over time
-** A prototype is imminent, 3 bugs blocking
-* Preliminary findings presentation available, email mbest@mozilla.com to setup a time.
-[[file:Bugzilla_Anthropology_Presentation.pdf | Bugzilla Anthropology Presentation]]
+* Progress on statistics side remains slow
+** A prototype is imminent, 3 bugs continue to block this progress although some progress has been made in recent days.
+** Once the database schema has been updated with the corrections, automated tests will need to be written to insure that data is trustworthy
+** A understanding of what security concerns there are remaining has been worked out with the security team and there is now a clear road map of what needs to happen to make the information public.
+*** Due to resource constraints, it's hard to get an ETA on when this work will be done.
+** Access to current experiments can be found at the following LDAP protected URL
+*** [https://metrics.mozilla.com/bugzilla-analysis-site/ Bugzilla Dashboard Prototypes]
+Initial presentation: [[file:Bugzilla_Anthropology_Presentation.pdf]]
 == Interviews ==
@@ Line 30: / Line 46: @@
 Below are the results of the interviews conducted so far.  These are the raw transcripts so they are unfiltered and the raw input that was given by the participants.  Summaries of this information will be coming but these will be useful for those that want to dig a little deeper.
+'''Summaries'''
+‎3 Page Summary on Interview Topic Categorization: [[file:BA_-_3_page_summary.pdf]]
+'''Interviews'''
 * [https://bugzillaanthropology.etherpad.mozilla.org/2 Ehsan Akhgari]
+* [https://bugzillaanthropology.etherpad.mozilla.org/24 Chris Atlee]
 * [https://bugzillaanthropology.etherpad.mozilla.org/5 David Baron]
+* [https://bugzillaanthropology.etherpad.mozilla.org/19 Juan Becerra]
+* [https://bugzillaanthropology.etherpad.mozilla.org/14 Christopher Blizzard]
 * [https://bugzillaanthropology.etherpad.mozilla.org/10 Kevin Brosnan]
 * [https://bugzillaanthropology.etherpad.mozilla.org/9 Matthew Brubeck]
+* [https://bugzillaanthropology.etherpad.mozilla.org/15 Asa Dotzler]
 * [https://bugzillaanthropology.etherpad.mozilla.org/6 Tyler Downer]
 * [https://bugzillaanthropology.etherpad.mozilla.org/7 Joe Drew]
+* [https://bugzillaanthropology.etherpad.mozilla.org/23 Axel Hecht]
 * [https://bugzillaanthropology.etherpad.mozilla.org/8 Brad Lassey]
 * [https://bugzillaanthropology.etherpad.mozilla.org/4 Christian Legnitto]
 * [https://bugzillaanthropology.etherpad.mozilla.org/1 David Mandelin]
+* [https://bugzillaanthropology.etherpad.mozilla.org/12 David Mason]
+* [https://bugzillaanthropology.etherpad.mozilla.org/18 Ed Morley]
 * [https://bugzillaanthropology.etherpad.mozilla.org/3 Jesse Ruderman]
+* [https://bugzillaanthropology.etherpad.mozilla.org/11 Gavin Sharp]
+* [https://bugzillaanthropology.etherpad.mozilla.org/16 Brian Warner]
+* [https://bugzillaanthropology.etherpad.mozilla.org/17 Boris Zbarsky]
+'''Write Ins'''
+* [https://bugzillaanthropology.etherpad.mozilla.org/21 Romaxa]
+* [https://bugzillaanthropology.etherpad.mozilla.org/20 Scoobidiver]
 == Statistics ==
 Gathering statistics on Bugzilla usage is a good way to provide an additional perspective that can help corroborate or question the information we gather via the interviews.  A tool has emerged that seems to provide some solid insight into this area.
-'''Note: Bug 704600 - Data has stopped collecting as of the 19th of November, there are also several bugs in the database that we are working to resolve with Metrics so this data should be double checked before making decisions against it until these issues are resolved.'''
 *[https://metrics.mozilla.com/pentaho/content/pentaho-cdf-dd/Render?solution=metrics&path=bugzilla/solr/QA&file=QA.wcdf QA Dashboard]