Auto-tools/Projects/Structured Logging: Difference between revisions
< Auto-tools | Projects
Jump to navigation
Jump to search
Chmanchester (talk | contribs) |
Chmanchester (talk | contribs) |
||
| Line 4: | Line 4: | ||
Goals: | Goals: | ||
* Support | * Support a range of output formats for test results | ||
* Eliminate complex and brittle regex based log parsing | * Eliminate complex and brittle regex based log parsing | ||
* Consolidate output processing and logging configuration between harnesses | * Consolidate output processing and logging configuration between harnesses | ||
Revision as of 05:35, 26 August 2014
This page is a high level account of the structured logging project providing links to many resources about the project. This page is a work in progress.
Background and Motivation
Goals:
- Support a range of output formats for test results
- Eliminate complex and brittle regex based log parsing
- Consolidate output processing and logging configuration between harnesses
Approach:
- Establish a common data format for test results and test harness diagnostics
- Populate data format in-harness with a logging API designed to output this format
- Log data from test harnesses as JSON rather than formatted strings
Implementation
Participant Systems
- System under test (Javascript, Java, C++)
- Python test harness
- Mozharness/Buildbot
- Tbpl/Treeherder
- Blobber
Logging APIs
The primary logging api for our data format is mozlog's structured module readthedocs. It is a part of Mozbase and is implemented in Python.
For data originating from JavaScript or Java, StructuredLog.jsm and StructuredLogger.java have been implemented to produce this data format. The mozlog API supports logging JSON directly from these producers with log_raw.
Status
- The tracking bug for this effort is bug916295.
- An etherpad with high-level status updates per test harness is maintained for the time being.