canmove, Confirmed users, Bureaucrats and Sysops emeriti
2,776
edits
Security/Reviews/BZ Elastic Search: Difference between revisions
Jump to navigation
Jump to search
Security/Reviews/BZ Elastic Search (view source)
Revision as of 19:44, 20 November 2013
, 20 November 2013no edit summary
(Created page with "{{SecReviewInfo |SecReview name=Store Bugzilla data in public ElasticSearch }} {{SecReview}} {{SecReviewActionStatus |SecReview action item status=None }}") |
No edit summary |
||
| Line 1: | Line 1: | ||
{{SecReviewInfo | {{SecReviewInfo | ||
|SecReview name=Store Bugzilla data in public ElasticSearch | |SecReview name=Store Bugzilla data in public ElasticSearch | ||
|SecReview target=<bugzilla> | |||
{ | |||
"id":"930081,930081" | |||
} | |||
</bugzilla> | |||
https://wiki.mozilla.org/Auto-tools/Projects/PublicES#SecReview_.2820_November_2013.2C_incomplete.29 | |||
http://people.mozilla.org/~klahnakoski/Achitecture.png | |||
Python code has been written that is responsible for | |||
* Extracting data directly from Bugzilla's database, | |||
* Transforming it to time-series data cube, and | |||
* Loading into publicly accessible ElasticSearch | |||
The known complications are: | |||
* Private bugs must not be included, and the history on those bugs must be removed from the historical record in ElasticSearch. | |||
* Private comments and private attachments must similarly be removed from the historical record. | |||
Additional Information: | |||
About: https://wiki.mozilla.org/Auto-tools/Projects/PublicES | |||
Code: https://github.com/klahnakoski/Bugzilla-ETL | |||
}} | |||
{{SecReview | |||
|SecReview feature goal=*dashboards | |||
*historical snapshots of bugs (point-in-time view) | |||
* Provide '''public''' fast cache of BZ data to: | |||
*# demonstrate current work | |||
*# allow community to build tools | |||
*#* https://github.com/okononen/dash | |||
*#* http://www.joshmatthews.net/bugsahoy/ | |||
*#* http://harthur.github.io/bzhome/ | |||
*#* http://pike.github.io/beta-dash/ | |||
*# allow community to analyze trends, patterns | |||
|SecReview alt solutions=* Tried to publicize the existing ES cluster information (private bugs with no comments or summary), but there was concern the CC list may reveal the bug's security category (https://bugzilla.mozilla.org/show_bug.cgi?id=823303) | |||
* Using the BZ-API directly requires sophisticated caching, which appears to stall attempts at making snappy dashboards. | |||
|SecReview solution chosen=* ElasticSearch is very fast | |||
* Direct DB access leverages existing code | |||
* Direct DB access puts no load on Bugzilla app | |||
* Proven to work with business intelligence queries, which demand fast aggregate data over thousands of bugs (https://wiki.mozilla.org/Bugzilla_Anthropology/2013-01-29) | |||
|SecReview threats considered=* Private bug data leaking into public cluster | |||
* ElasticSearch was not meant for direct public access, proxy added (https://bugzilla.mozilla.org/show_bug.cgi?id=879833) | |||
|SecReview threat brainstorming=* Elastic Search index tampering (delete, rename, etc) | |||
* ES Script injection (MVEL) | |||
* DOS | |||
* Bugs that are changed from public to private (aka, how often is data refreshed?) | |||
* Data exfiltration via bug posting | |||
}} | }} | ||
{{SecReviewActionStatus | {{SecReviewActionStatus | ||
|SecReview action item status=None | |SecReview action item status=None | ||
}} | }} | ||