Auto-tools/Projects/OrangeFactor/2010-11-03
War on Orange Status Meeting, Nov 3, 2010
Anurag and Daniel joined us today, and we discussed the eventual setup w.r.t. logs and the database.
Log processing
A setup for handling the logs at different stages of processing was proposed as follows:
1 - a script saves new buildbot logs to a directory, say 'incoming'
2 - another script watches for files to appear in 'incoming', then copies them to another directory called 'parse'. The log parser is invoked for this log (by our script? or flume?), and the output sent to stdout, which somehow flume will be watching, and which it will transport to the Hive DB. Finally, the log is moved to another directory, 'sendlog'.
3 - another script watches for files to appear in 'sendlog', and outputs the entire (uncompressed) log to stdout, with the following format (' \t ' represents a tab in the output):
repo \t platform \t debug-or-opt \t builddate \t testsuite \t [loglines]
where [loglines] is a JSON array of log lines (one log line per element in the array)
Afterwards, the log is moved to a 'processed' directory.
The 'repo', 'platform', 'debug|opt', 'builddate' and 'suitename' properties above are generated from the original location of the log on http://stage.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds, e.g., the output for
mozilla-central-macosx64-debug/1288731175/mozilla-central_snowleopard-debug_test-crashtest-build94.txt.gz
would be:
mozilla-central macosx64 debug 1288731175 crashtest [loglines]
Each of the these properties will wind up being columns in the logdata database, so we should be able associate these logs with the parsed log data. If the parsed log data includes indexes into these logs, then we'll be able to retrieve a portion of the log that represents the execution of any individual test.
Orange Factor
Jeff has added a 'Testing: Orange Factor' component in bugzilla, and plans on filing a lot of bugs with ideas for improving the site.
Joel has moved the website to brasstacks, and it has increased in speed mightily! He's changed the main display to show a daily orange factor calculation, rather than a cumulative average. Jeff has some ideas of how to apply some smoothing algorithms to this so we can reduce the spikiness of the graph. Mark plans to update the page titles to be distinct.
We discussed the fact that the new db will give us data on test failures, while the current web site shows data on bugs. Once we switch to the new db, how will we associate failures with bugs?
We'll investigate adding a little script to the pushlog webpage which sends our db an XHR each time a failure is starred. Another option is to use the pushlog algorithm which guesses which bug a failure is related to.