Socorro:ElasticSearch API

From MozillaWiki
Revision as of 23:54, 4 May 2011 by Adrigau (talk | contribs) (→‎API Spec: Adding more reports types.)
Jump to navigation Jump to search

Middleware API for ElasticSearch

This is a draft of the new API for querying ElasticSearch through the middleware API of Socorro.

The Middleware API aims to separate the front-end from the back-end by providing an interface to access the data. By doing so, the front-end will not have to care about the storage system, and will retrieve data from Hbase, PostgreSQL or ElasticSearch in a consistent and simple way, through our REST API.

The API is separated in several categories / entry points:

  • /query
  • /search
  • /report
  • /crash
  • /stats

This categories are explained below.

Query

Description

Low level query, just sends a JSON query to ES directly, and returns the result of this query.

API Spec

HTTP request: POST
Data: JSON query to send to ElasticSearch
URI: /query/[(types)/]

  • types: Types of data we are looking into. If omitted, default value is _all. Several types can be specified, separated by a + symbol.

Search

Description

Searches for some terms in ES.

API Spec

HTTP request: GET
URI: /search/(types)/for/(terms)/in/(fields)/product/(product)/version/(version)/os/(os_name)/from/(from_date)/to/(to_date)/(optional_parameters)

  • types: Type of data we are looking into. Can be set to _all to search into all types. Several types can be specified, separated by a + symbol.
  • terms: Terms we are search for. Each term must be URL encoded. Several terms can be specified, separated by a + symbol.
  • fields: Fields we are searching in. Several fields can be specified, separated by a + symbol.
  • product: The product we are interested in. (e.g. Firefox, Fennec, Thunderbird... )
  • version: Version of the product. Can be set to _all to search into all versions.
  • os_name: Name of the Operating System. (e.g. Windows, Mac, Linux... )
  • from_date: Search for crashes that happened after this date.
  • to_date: Search for crashes that happened before this date.

Optional parameters:

This request has some optional parameters that can be omitted. Any omitted parameter is not used while querying ES (meaning, there is no default value for this parameters). This parameters can be set in any order, and you can use only some of them if you want to.

The complete URI is as follow: /search/(types)/for/(terms)/in/(fields)/product/(product)/version/(version)/os/(os_name)/from/(from_date)/to/(to_date)/branches/(branches)/search_mode/(search_mode)/reason/(crash_reason)/build/(build_id)/report_process/(report_process)/report_type/(report_type)/plugin_in/(plugin_in)/plugin_search_mode/(plugin_search_mode)/plugin_term/(plugin_term)

  • branches: Several branches can be specified, separated by a + symbol.
  • search_mode: Set how to search. Can be either 'is_exactly', 'contains' or 'start_with'.
  • crash_reason: Restricts search to crashes caused by this reason.
  • build_id: Restricts search to crashes that happened on a product with this build ID.
  • report_process: Can be 'any', 'browser' or 'plugin'.
  • report_type: Can be 'any', 'crash' or 'hang'.
  • plugin_in: Search for a plugin in this field.
  • plugin_search_mode: How to search for this plugin.
  • plugin_term: Terms to search for. Several terms can be specified, separated by a + symbol.

Report

Description

Get a specific report.

API Spec

HTTP request: GET
URI: /report/(report_name)/(report_parameters)

Top Changers by Signature

URI: /top_changers_by_signature/product/(product)/version/(version)/from/(from_date)/to/(to_date)/

  • product: The product we are interested in. (e.g. Firefox, Fennec, Thunderbird... )
  • version: Version of the product.
  • from_date: Only crashes that happened after this date.
  • to_date: Only crashes that happened before this date.

Top Crashers by Signature

URI: /top_crashers_by_signature/product/(product)/version/(version)/from/(from_date)/to/(to_date)/

  • product: The product we are interested in. (e.g. Firefox, Fennec, Thunderbird... )
  • version: Version of the product.
  • from_date: Only crashes that happened after this date.
  • to_date: Only crashes that happened before this date.

Top Crashers by URL

URI: /top_crashers_by_url/...

Top Crashers by Domain

URI: /top_crashers_by_domain/...

Top Crashers by Topsite

URI: /top_crashers_by_topsite/...

Crash

Description

Searches a crash by it's OOID and returns it. This query is not supposed to use ES but HBase directly.

API Spec

HTTP request: GET
URI: /crash/(crash_id)

  • crash_id: Unique identifier of the crash to retrieve.


Stats

Description

This is a proposition.

Get some statistics around the data. E.g. counting by OS, by product, by ADU, by build... The difference with report is that stats only send back numeric data, counting through the entire data set or in a certain date range.

Retrieved from "https://wiki.allizom.org/index.php?title=Socorro:ElasticSearch_API&oldid=305705"