BMO/ElasticSearch: Difference between revisions
< BMO
Jump to navigation
Jump to search
Klahnakoski (talk | contribs) m (→Let's Use It!) |
Klahnakoski (talk | contribs) (easier reading) |
||
| (4 intermediate revisions by 2 users not shown) | |||
| Line 7: | Line 7: | ||
== Justification == | == Justification == | ||
The BMO/ES cluster is important for managing many of Mozilla's projects: It | The BMO/ES cluster is important for managing many of Mozilla's projects: It is an [https://wiki.mozilla.org/Projects/ActiveData active data] system that underlies a multitude of dashboards to track various project metrics. It is capable of providing both high level aggregates over the million+ issues, providing detailed historical views into those issues, and it provides that information ''fast''. | ||
= Let's Use It! = | = Let's Use It! = | ||
BMO/ES is a service, which means you can use it right now! You probably already have your browser open, so | BMO/ES is a service, which means you can use it right now! You probably already have your browser open, so let's send it queries: | ||
==Examples== | ==Examples== | ||
| Line 17: | Line 17: | ||
:[http://people.mozilla.org/~klahnakoski/modevmetrics/Tutorial02-Comments.html Minimum required to pull comments] [https://github.com/klahnakoski/MoDevMetrics/blob/dev/html/Tutorial02-Comments.html on Github] | :[http://people.mozilla.org/~klahnakoski/modevmetrics/Tutorial02-Comments.html Minimum required to pull comments] [https://github.com/klahnakoski/MoDevMetrics/blob/dev/html/Tutorial02-Comments.html on Github] | ||
:[http://people.mozilla.org/~klahnakoski/modevmetrics/Tutorial03-Bug-Count.html Simple Bug Count] [https://github.com/klahnakoski/MoDevMetrics/blob/dev/html/Tutorial03-Bug-Count.html on Github] | :[http://people.mozilla.org/~klahnakoski/modevmetrics/Tutorial03-Bug-Count.html Simple Bug Count] [https://github.com/klahnakoski/MoDevMetrics/blob/dev/html/Tutorial03-Bug-Count.html on Github] | ||
:[http://people.mozilla.org/~klahnakoski/modevmetrics/Tutorial04-Private.html Can you reach the private cluster?] [https://github.com/klahnakoski/MoDevMetrics/blob/dev/html/Tutorial04-Private.html on Github] | |||
==Basic walk-through== | ==Basic walk-through== | ||
:https://github.com/klahnakoski/Qb/blob/master/docs/BZ_Tutorial.md | :https://github.com/klahnakoski/Qb/blob/master/docs/BZ_Tutorial.md | ||
| Line 31: | Line 33: | ||
* '''IRC:''' ekyle@irc.mozilla.org (in #ateam and #bteam) | * '''IRC:''' ekyle@irc.mozilla.org (in #ateam and #bteam) | ||
* '''email:''' klahnakoski@mozilla.com | * '''email:''' klahnakoski@mozilla.com | ||
== Good First Bugs == | == Good First Bugs == | ||
| Line 72: | Line 72: | ||
* [https://wiki.mozilla.org/BMO/ElasticSearch/History#New_Design Architecture] included in part of the adventures | * [https://wiki.mozilla.org/BMO/ElasticSearch/History#New_Design Architecture] included in part of the adventures | ||
* A [http://people.mozilla.org/~klahnakoski/docs/Bugs%20in%20ES.pdf ppt presentation] to show what you can build using this cluster as a [http://en.wikipedia.org/wiki/Data_mart datamart]. | * A [http://people.mozilla.org/~klahnakoski/docs/Bugs%20in%20ES.pdf ppt presentation] to show what you can build using this cluster as a [http://en.wikipedia.org/wiki/Data_mart datamart]. | ||
[[Category:BMO]] | |||
Latest revision as of 17:55, 15 December 2015
A Public ElasticSearch Cluster with Bugzilla Data
Objective
Provide the public with an easy-to-query historical view of Bugzilla bugs.
Justification
The BMO/ES cluster is important for managing many of Mozilla's projects: It is an active data system that underlies a multitude of dashboards to track various project metrics. It is capable of providing both high level aggregates over the million+ issues, providing detailed historical views into those issues, and it provides that information fast.
Let's Use It!
BMO/ES is a service, which means you can use it right now! You probably already have your browser open, so let's send it queries:
Examples
- Minimum required to hit ES cluster on Github
- Minimum required to pull comments on Github
- Simple Bug Count on Github
- Can you reach the private cluster? on Github
Basic walk-through
Code
- Backend ETL Code: https://github.com/klahnakoski/Bugzilla-ETL
- Frontend Examples: https://github.com/klahnakoski/MoDevMetrics/tree/dev
Hacking/Contributors
This project needs your help! If you love data munging and dealing with complexity; and you get satisfaction that people will build atop your hard work every day, then this project is for you!
Contacts
- IRC: ekyle@irc.mozilla.org (in #ateam and #bteam)
- email: klahnakoski@mozilla.com
Good First Bugs
No results.
0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);
Big bugs
| ID | Summary | Status | Resolution | Assigned to |
|---|---|---|---|---|
| 997228 | Renaming flags will destroy bug history for those flag values | NEW | ||
| 1125904 | Blocked/blocked_by/depends_on is too confusing | NEW | ||
| 1197813 | [BZ ETL] Tracking flags changed corrupt bug history | NEW | ||
| 1201064 | cf_blocking_loop dropped (renamed), not reflected in history | NEW |
4 Total; 4 Open (100%); 0 Resolved (0%); 0 Verified (0%);
Other Reading
- Adventures in initial BMO/ES development
- Architecture included in part of the adventures
- A ppt presentation to show what you can build using this cluster as a datamart.