Confirmed users
513
edits
BMO/ElasticSearch: Difference between revisions
→SecReview (20 November 2013)
Klahnakoski (talk | contribs) |
Klahnakoski (talk | contribs) |
||
| Line 80: | Line 80: | ||
In the end we have a high speed ETL solution that is easy to install and execute. There are plenty of improvements that can be made, and definitely in the area of more threads and more multiple processes. But those can wait while we deploy. | In the end we have a high speed ETL solution that is easy to install and execute. There are plenty of improvements that can be made, and definitely in the area of more threads and more multiple processes. But those can wait while we deploy. | ||
= SecReview (20 November 2013) = | == SecReview (20 November 2013) == | ||
== Introduction == | === Introduction === | ||
Public Bugzilla data in a publicly accessible ElasticSearch cluster! | Public Bugzilla data in a publicly accessible ElasticSearch cluster! | ||
| Line 94: | Line 94: | ||
* '''History''' https://wiki.mozilla.org/index.php?title=Bugzilla_Anthropology/2013-01-29 | * '''History''' https://wiki.mozilla.org/index.php?title=Bugzilla_Anthropology/2013-01-29 | ||
=== Goal of Feature, what is trying to be achieved (problem solved, use cases, etc) === | ==== Goal of Feature, what is trying to be achieved (problem solved, use cases, etc) ==== | ||
* Provide '''public''' fast cache of BZ data to: | * Provide '''public''' fast cache of BZ data to: | ||
*# demonstrate current work | *# demonstrate current work | ||
| Line 104: | Line 104: | ||
*# allow community to analyze trends, patterns | *# allow community to analyze trends, patterns | ||
=== What solutions/approaches were considered other than the proposed solution? === | ==== What solutions/approaches were considered other than the proposed solution? ==== | ||
* Tried to publicize the existing ES cluster information (private bugs with no comments or summary), but there was concern the CC list may reveal the bug's security category (https://bugzilla.mozilla.org/show_bug.cgi?id=823303) | * Tried to publicize the existing ES cluster information (private bugs with no comments or summary), but there was concern the CC list may reveal the bug's security category (https://bugzilla.mozilla.org/show_bug.cgi?id=823303) | ||
* Using the BZ-API directly requires sophisticated caching, which appears to stall attempts at making snappy dashboards. | * Using the BZ-API directly requires sophisticated caching, which appears to stall attempts at making snappy dashboards. | ||
=== Why was this solution chosen? === | ==== Why was this solution chosen? ==== | ||
* ElasticSearch is very fast | * ElasticSearch is very fast | ||
* Direct DB access leverages existing code | * Direct DB access leverages existing code | ||
| Line 114: | Line 114: | ||
* Proven to work with business intelligence queries, which demand fast aggregate data over thousands of bugs (https://wiki.mozilla.org/Bugzilla_Anthropology/2013-01-29) | * Proven to work with business intelligence queries, which demand fast aggregate data over thousands of bugs (https://wiki.mozilla.org/Bugzilla_Anthropology/2013-01-29) | ||
=== Any security threats already considered in the design and why? === | ==== Any security threats already considered in the design and why? ==== | ||
* Private bug data leaking into public cluster | * Private bug data leaking into public cluster | ||
* ElasticSearch was not meant for direct public access, proxy added (https://bugzilla.mozilla.org/show_bug.cgi?id=879833) | * ElasticSearch was not meant for direct public access, proxy added (https://bugzilla.mozilla.org/show_bug.cgi?id=879833) | ||