View source for EngineeringProductivity/Projects/Treeherder/Meetings/2013-01-23

= Design Notes =
*[https://etherpad.mozilla.org/MyhQ3VxUHC etherpad] from the meeting
== Build Objects ==
* Define build base object 
** Not specific to builbot per se but would have everything we need based on what ever kind of build we are working with. However the data we are working with now is really messy and inconsistent. The structure will need to be pretty flexible to accomodate this.
* One option for flexibility would be to have an object store that accepts more attributes that aren't on all objects (the set of required attributes would be small) -- similar to what we did with datazilla.  The index of the attributes could be project specific.
* There will be a core set of attributes that all objects have to have but the specifics will differ among various projects.
* Could put a proxy inside the build network so that the build system would not have to use auth to talk to this tool but things outside the build network would.  The proxy could implement OAuth for the build network to simplify things.

== Test Objects ==
* Test data will be stored in a dedicated objectstore as well and then datazilla/orangefactor/graphserver will query this. This will also break the synchonicity between the downstream systems and the build automation.  Putting it in one object store will make it easier to scale and to make it more reliable.
* Generation of Test Objects, in many cases, is done through log parsing.  This part of the project has hard dependencies on other log parsing clean up projects.
* Test objects should not be limited to data from logs. If you could have raw log, parsed log, and references to binary locations, screen shots, recordings during tests, etc all could be keys inside results dictionary.
* Parsed logs would be all the data from the log - python lib that the slaves could use to parse the log before upload but the raw log would also be uploaded as well.
** Note: Not sure we need to store the raw log separately, could just store a pointer to it. How does the storage of raw logs currently work? How long do we keep them for?
* Some of the test automation frameworks generate their own summaries and buildbot generates its own summaries on top of that, this should be cleaned up as part of this effort or alongside it.
** This should be done in mozharness not in buildbot
* The build object can have any number of test objects associated with it.
* There are logs outside the tests - the buildbot master log etc.

== Pushlog Objects ==
* How do we maintain the ordered pushlog?
* The current way to do that is to review the pushes from hg and then combine that with data from the build system and generate the pushlog
* would be great to have a webservice that gives you a unified view of the pushlog so that all downstream apps can use it
* we need this for git not just hg

== Scheduled Job Objects (valgrind, releases etc...) ==
* There needs to be some way to see and understand what the status is of these auxiliary build/test jobs that run more periodically - valgrind, QA update tests etc.
* These could be managed with a build object type attribute

== Machine State Objects ==
* List of machines and their current state for each build system

== Putting it All Together ==
* The structure of how tests and builds are related and even "jobs" depend on other jobs are different and the way that we handle these things is complex - for releases for instance we "fan in" w.r.t. various jobs that must complete before we do the next job.  For m-c we "fan out" where a build finishes and then the tests are run for that build, etc.
* Building a deep correlation with builds and tests into tbpl v2 might bite us or constrain us. So we need to think about supporting the release automation and l10n builds
** Note: not sure if this would bite us if build objects and test objects are stored in separate object stores with no constraints on when test data gets populated.
* We need to fit in l10n somewhere
** For l10N we key off an english build and then do a localized build
** we also generate a one-off repack for l10ners from older nightly english builds
** perhaps we handle this with generic relations between job objects and dependent jobs
** this would also help manage the scheduled jobs and things that would fit the non-push log based approach.
* TODO: Is supporting release automation for this out of scope? We would need some way to fold it in in the future, but not clear how much we can do early.
** For instance the socorro team would love more information on releases right now they scrape ftp and pray. :)

== UI Design ==
* We will actually get to this some day... no really...

== Action Items ==
* Design build objects