Latest revision as of 17:55, 15 December 2015

A Public ElasticSearch Cluster with Bugzilla Data

Objective

Provide the public with an easy-to-query historical view of Bugzilla bugs.

Justification

The BMO/ES cluster is important for managing many of Mozilla's projects: It is an active data system that underlies a multitude of dashboards to track various project metrics. It is capable of providing both high level aggregates over the million+ issues, providing detailed historical views into those issues, and it provides that information fast.

Let's Use It!

BMO/ES is a service, which means you can use it right now! You probably already have your browser open, so let's send it queries:

Examples

Minimum required to hit ES cluster on Github

Minimum required to pull comments on Github

Simple Bug Count on Github

Can you reach the private cluster? on Github

Basic walk-through

https://github.com/klahnakoski/Qb/blob/master/docs/BZ_Tutorial.md

Code

Backend ETL Code: https://github.com/klahnakoski/Bugzilla-ETL

Frontend Examples: https://github.com/klahnakoski/MoDevMetrics/tree/dev

Hacking/Contributors

This project needs your help! If you love data munging and dealing with complexity; and you get satisfaction that people will build atop your hard work every day, then this project is for you!

Contacts

IRC: ekyle@irc.mozilla.org (in #ateam and #bteam)
email: klahnakoski@mozilla.com

Good First Bugs

No results.

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);

Big bugs

Full Query

ID	Summary	Status
997228	Renaming flags will destroy bug history for those flag values	NEW
1125904	Blocked/blocked_by/depends_on is too confusing	NEW
1197813	[BZ ETL] Tracking flags changed corrupt bug history	NEW
1201064	cf_blocking_loop dropped (renamed), not reflected in history	NEW

4 Total; 4 Open (100%); 0 Resolved (0%); 0 Verified (0%);

@@ Line 7: / Line 7: @@
 == Justification ==
-The BMO/ES cluster is important for managing many of Mozilla's projects:  It  is the underlying [https://wiki.mozilla.org/Projects/ActiveData active data] system that is used by a multitude of dashboards to track various project metrics.  It is capable of providing both high level aggregates over the million+ issues, providing detailed historical views into those issues, and it provides that information ''fast''.
+The BMO/ES cluster is important for managing many of Mozilla's projects:  It is an [https://wiki.mozilla.org/Projects/ActiveData active data] system that underlies a multitude of dashboards to track various project metrics.  It is capable of providing both high level aggregates over the million+ issues, providing detailed historical views into those issues, and it provides that information ''fast''.
-== Let's Use It! ==
+= Let's Use It! =
-BMO/ES is a service, which means you can use it right now!  You probably already have your browser open, so lets send it queries:
+BMO/ES is a service, which means you can use it right now!  You probably already have your browser open, so let's send it queries:
-'''Examples'''
+==Examples==
 :[http://people.mozilla.org/~klahnakoski/modevmetrics/Tutorial01-Minimum.html Minimum required to hit ES cluster] [https://github.com/klahnakoski/MoDevMetrics/blob/dev/html/Tutorial01-Minimum.html on Github]
 :[http://people.mozilla.org/~klahnakoski/modevmetrics/Tutorial02-Comments.html Minimum required to pull comments] [https://github.com/klahnakoski/MoDevMetrics/blob/dev/html/Tutorial02-Comments.html on Github]
 :[http://people.mozilla.org/~klahnakoski/modevmetrics/Tutorial03-Bug-Count.html Simple Bug Count] [https://github.com/klahnakoski/MoDevMetrics/blob/dev/html/Tutorial03-Bug-Count.html on Github]
-'''Basic walk-through'''
+:[http://people.mozilla.org/~klahnakoski/modevmetrics/Tutorial04-Private.html Can you reach the private cluster?] [https://github.com/klahnakoski/MoDevMetrics/blob/dev/html/Tutorial04-Private.html on Github]
+==Basic walk-through==
 :https://github.com/klahnakoski/Qb/blob/master/docs/BZ_Tutorial.md
-'''Code'''
+==Code==
 :Backend ETL Code: https://github.com/klahnakoski/Bugzilla-ETL
 :Frontend Examples: https://github.com/klahnakoski/MoDevMetrics/tree/dev
+= Hacking/Contributors =
-== Hacking/Contributors ==
 This project needs your help!  If you love data munging and dealing with complexity; and you get satisfaction that people will build atop your hard work every day, then this project is for you!
@@ Line 33: / Line 34: @@
 * '''email:''' klahnakoski@mozilla.com
-I know hard this job is, and how little it gets noticed, maybe we can hang out on IRC and commiserate together!
+== Good First Bugs ==
+<div>
+<bugzilla>
+{
+"include_fields": "id, summary, status, resolution, assigned_to",
+"resolution":"---",
+"f1":"blocked",
+"o1":"equals",
+"v1":959670,
+"f2":"status_whiteboard",
+"o2":"anywordssubstr",
+"v2":"[good first bug]"
+}
+</bugzilla>
+</div>
+== Big bugs ==
+<div>
+<bugzilla>
+{
+"include_fields": "id, summary, status, resolution, assigned_to",
+"resolution":"---",
+"f1":"blocked",
+"o1":"equals",
+"v1":959670,
+"f2":"status_whiteboard",
+"o2":"nowordssubstr",
+"v2":"[meta]"
+}
+</bugzilla>
+</div>
 == Other Reading ==
-* [https://wiki.mozilla.org/BMO/ElasticSearch/History Annals of adventures in initial BMO/ES development] - yet another web timesink!
+* [https://wiki.mozilla.org/BMO/ElasticSearch/History Adventures in initial BMO/ES development]
+* [https://wiki.mozilla.org/BMO/ElasticSearch/History#New_Design Architecture] included in part of the adventures
 * A [http://people.mozilla.org/~klahnakoski/docs/Bugs%20in%20ES.pdf ppt presentation] to show what you can build using this cluster as a [http://en.wikipedia.org/wiki/Data_mart datamart].
+[[Category:BMO]]

BMO/ElasticSearch: Difference between revisions