|
|
| (40 intermediate revisions by 7 users not shown) |
| Line 1: |
Line 1: |
| == Overview == | | == About == |
| | [https://treeherder.mozilla.org/ Treeherder] is a reporting dashboard for checkins to Mozilla projects (for example, [https://developer.mozilla.org/en-US/docs/mozilla-central mozilla-central]). It allows users to see the results of automatic builds and their respective tests. Treeherder also provides a rich set of APIs that can be used by other projects interested in this information. |
|
| |
|
| [https://treeherder.mozilla.org/ Treeherder] is a reporting dashboard for checkins to Mozilla projects (for example, [https://developer.mozilla.org/en-US/docs/mozilla-central mozilla-central] or [https://wiki.mozilla.org/Gaia gaia]). It allows users to see the results of automatic builds and their respective tests. Treeherder also provides a rich set of API's that can be used by other projects interested in this information.
| | Treeherder is the successor to [[Sheriffing/TBPL|TBPL]]. |
|
| |
|
| Treeherder is the successor to [[Sheriffing/TBPL|TBPL]]. | | For tracking performance data, see Treeherder's sister project, [[EngineeringProductivity/Projects/Perfherder|Perfherder]]. |
| | |
| | For tracking intermittent test failures, see [[EngineeringProductivity/Projects/IntermittentFailuresView|Intermittent Failures View]]. |
|
| |
|
| == Contributing == | | == Contributing == |
| To make UI changes, in many cases you only need to perform a very simple setup running a local webserver pointing at the production instance, described [https://treeherder-ui.readthedocs.org/en/latest/installation.html here]. If you wish to hack on the backend, or the UI and backend together, you will instead need to set up both the [https://github.com/mozilla/treeherder-service treeherder-ui] and [https://github.com/mozilla/treeherder-service treeherder-service] Github repos as a pair, following [https://treeherder-service.readthedocs.org/en/latest/installation.html these steps]. | | To make UI-only changes, you only need to perform a very simple setup running a local webserver pointing at the production backend, described [https://treeherder.readthedocs.io/installation.html#ui-development here]. If you wish to hack on the backend, or the UI and backend together, you will instead need to set up a Vagrant environment, using [https://treeherder.readthedocs.io/installation.html#server-and-full-stack-development these steps]. |
| * A-Team Bootcamp: Best practices for working on A-Team projects of which Treeherder is one. If you're new to Mozilla or the A-Team, please read this guide thoroughly before proceeding.
| | |
| * Good first bugs for new developers: [http://www.joshmatthews.net/bugsahoy/?reporting=1 Bugs Ahoy] | | * [https://ateam-bootcamp.readthedocs.io/en/latest/ A-Team Bootcamp]: Best practices for working on A-Team projects (of which Treeherder is one), including valuable information on using Git and Bugzilla. If you're new to Mozilla or the A-Team, please read this guide thoroughly before proceeding. |
| * Bugzilla: [https://bugzilla.mozilla.org/enter_bug.cgi?product=Tree+Management&component=Treeherder File a bug] / [https://bugzilla.mozilla.org/buglist.cgi?product=Tree+Management&f1=component&v1=Treeherder&o1=substring&resolution=--- All open bugs] / [https://bugzilla.mozilla.org/buglist.cgi?order=Bug%20Number&resolution=FIXED&chfieldto=Now&chfield=resolution&chfieldfrom=-14d&chfieldvalue=FIXED&f1=component&v1=Treeherder&o1=substring Recently fixed] / [[#Bug_Tables|Bug tables]] | | * Good first bugs for new developers: [https://codetribute.mozilla.org/projects/treeherder Codetribute] |
| * Chat on IRC: [irc://irc.mozilla.org/treeherder #treeherder] / [http://logs.glob.uno/?c=treeherder Channel logs] (to learn about IRC see [[IRC]])
| | * Issue tracker: [https://bugzilla.mozilla.org/enter_bug.cgi?product=Tree+Management&component=Treeherder Report a bug] / [https://bugzilla.mozilla.org/query.cgi?query_format=advanced&product=Tree+Management&f1=component&o1=substring&v1=Treeherder&resolution=--- search open bugs] |
| * Meetings: [[Auto-tools/Projects/Treeherder/Meetings|Notes & dial-in details]]
| | * Mozilla Treeherder instances: [https://treeherder.allizom.org Staging] / [https://treeherder.mozilla.org Production] / [https://whatsdeployed.io/s-dqv Deployed changes info] |
| * Mozilla Treeherder instances: [http://treeherder-dev.allizom.org Dev] / [https://treeherder.allizom.org Staging] / [https://treeherder.mozilla.org Production] ([https://mana.mozilla.org/wiki/display/websites/treeherder.mozilla.org mana page]) | |
|
| |
|
| == Source and Docs == | | == Source and Docs == |
| * UI: [https://github.com/mozilla/treeherder-ui Source] / [https://treeherder-ui.readthedocs.org Docs] | | * UI & backend: [https://github.com/mozilla/treeherder Source] / [https://treeherder.readthedocs.io Docs] |
| * Backend: [https://github.com/mozilla/treeherder-service Source] / [https://treeherder-service.readthedocs.org Docs] / [http://treeherder-dev.allizom.org/docs/ API Docs]
| |
| * Treeherder data submission clients: | | * Treeherder data submission clients: |
| ** Python: [https://github.com/mozilla/treeherder-client Source] / [https://github.com/mozilla/treeherder-client/blob/master/README.md Readme] | | ** Pulse: [https://hg.mozilla.org/automation/mozillapulse/ Source] / [http://treeherder.readthedocs.io/submitting_data.html#submitting-using-pulse docs] |
| ** NodeJS: [https://github.com/mozilla/treeherder-node Source] / [https://github.com/mozilla/treeherder-node/blob/master/README.md Readme]
| | ** (Python and Node submission support has been removed.) |
| | |
| == Roadmap ==
| |
| ==== Milestone M1: Functional Prototype ====
| |
| * Deliver a functional prototype that sheriffs can use for E2E testing.
| |
| * Status: {{done|}} delivered 2014 Q1.
| |
| ==== Milestone M2: Production Deployment ====
| |
| * Complete a security review and deploy into production with IT.
| |
| * Status: {{done|}} delivered 2014 Q2.
| |
| ==== Milestone M3: Sheriff Transition from TBPL ====
| |
| * Work with the sheriffs to address bugs and workflow issues so that Treeherder is a viable alternate for TBPL for their workflow, and transition the sheriffs to using Treeherder for their day-to-day work. Bugs: see dependents of {{bug|1030636}}.
| |
| * Status: {{done|}} delivered 2014 Q3 ([https://groups.google.com/forum/#!topic/mozilla.dev.tree-management/WSQY2EYgxz8 newsgroup post]).
| |
| | |
| ==== Milestone M4: Make TBPL redundant for all users ====
| |
| * Work to implement other developer use cases, refine sheriffing work flows, and make TBPL redundant for all of its users. | |
| * Status: {{done|}} delivered 2015 Q1 ([https://groups.google.com/forum/#!topic/mozilla.dev.tree-management/U4itH2b711Q newsgroup post]). | |
|
| |
|
| ==== Milestone M5: Performance Data ==== | | == Getting in touch == |
| * We want to obsolete Graphserver and Datazilla, and make Treeherder the dashboard that is used to visualize performance data generated by Talos and other sources. This will be comprised of at least the following: | | * Chat on IRC: [irc://irc.mozilla.org/treeherder #treeherder] / [[IRC|learn about IRC]] / [https://mozilla.logbot.info/treeherder channel logs] |
| ** A performance panel in pushlog view that can be used to see details about the performance data submitted for a particular push, plus recent antecedents. | | * Mailing list: [https://lists.mozilla.org/listinfo/tools-treeherder tools-treeherder] (or [https://groups.google.com/forum/#!forum/mozilla.tools.treeherder via Google groups]) |
| ** A performance summary tab that gives a high-level overview by branch, platform, and suite.
| | * Weekly meetings: [[EngineeringProductivity/Projects/Treeherder/Meetings|Notes & dial-in details]] |
| * Status: {{proposed|}} detailed planning to take place in late 2014 Q4, with delivery in 2015 Q1. | |
|
| |
|
| ==== Possible Future Tasks ==== | | == What we're working on == |
| * Migrating to Heroku
| | Assigned Treeherder bugs that have been modified in the last month. |
| * Using virtualenv/peep for more reliable package management
| | [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=%3Atreeherder+-assignee%3Anobody%40mozilla.org+delta_ts%3E-1m View on Bugzilla] |
| * Merging the treeherder-ui and treeherder-service repos
| | <bugzilla stats="hide"> |
| * Removing the objectstore
| |
| * Splitting of oauth credentials from the datasource table
| |
| * Improving UI performance/memory consumption
| |
| * Storing job artefacts in S3
| |
| * Improving task debug-ability (eg more logging, log aggregation)
| |
| * Making it easier for new contributors to setup/perform common tasks
| |
| * Adding more UI tests
| |
| * Refactoring of jobs.py, including splitting out refdata
| |
| * Moving away from datasource (ie using Django ORM)
| |
| * Creating a log viewer UI for displaying json logs
| |
| * Intermittent bug filer
| |
| * Annotation of backouts, tree closures
| |
| * Better performance graphs
| |
| * Integration of performance alerts
| |
| * Integration of OrangeFactor, GoFaster dashboards
| |
| * Create a display for code coverage data
| |
| * Additional metrics and visualizations
| |
| | |
| == Bug Tables ==
| |
| === [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=%3Atreeherder+-assignee%3Anobody%40mozilla.org Assigned] ===
| |
| A summary of Treeherder bugs that are being worked on at the moment.
| |
| <bugzilla>
| |
| {
| |
| "component": "Treeherder", "component_type": "contains",
| |
| "resolution": "---",
| |
| "email1": "nobody@mozilla.org", "email1_type": "not_equals", "email1_assigned_to": "1",
| |
| "include_fields": "id,priority,component,summary,assigned_to",
| |
| "order": "priority,assigned_to"
| |
| }
| |
| </bugzilla>
| |
| === [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=%3Atreeherder+P1+assignee%3Anobody%40mozilla.org Unassigned P1] ===
| |
| All open P1 Treeherder bugs that do not have an assignee. Ideally these are the bugs that should be worked on next, once the currently assigned bugs are fixed.
| |
| <bugzilla>
| |
| {
| |
| "component": "Treeherder", "component_type": "contains",
| |
| "resolution": "---",
| |
| "email1": "nobody@mozilla.org", "email1_type": "equals", "email1_assigned_to": "1",
| |
| "priority": "P1",
| |
| "include_fields": "id,component,summary",
| |
| "order": "component"
| |
| }
| |
| </bugzilla>
| |
| === [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=%3Atreeherder+priority%3A--+assignee%3Anobody%40mozilla.org Needing Triage] ===
| |
| All bugs that do not have a priority set and are unassigned (excluding Perfherder). These are ideally triaged weekly.
| |
| <bugzilla>
| |
| {
| |
| "quicksearch": ":treeherder -:perfherder priority:-- assignee:nobody@mozilla.org",
| |
| "include_fields": "id,component,summary",
| |
| "order": "component"
| |
| }
| |
| </bugzilla>
| |
| === [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=%3Atreeherder All open bugs] ===
| |
| <bugzilla>
| |
| {
| |
| "component": "Treeherder", "component_type": "contains",
| |
| "resolution": "---",
| |
| "include_fields": "id,priority,component,summary,assigned_to"
| |
| }
| |
| </bugzilla> | |
| === [https://bugzilla.mozilla.org/buglist.cgi?order=Bug%20Number&resolution=FIXED&chfieldto=Now&chfield=resolution&chfieldfrom=-14d&chfieldvalue=FIXED&f1=component&v1=Treeherder&o1=substring Recently fixed] ===
| |
| Bugs changed to "Resolved FIXED" in the last 14 days.
| |
| <bugzilla>
| |
| { | | { |
| "component": "Treeherder", "component_type": "contains", | | "component": "Treeherder", "component_type": "contains", |
| "resolution": "FIXED", | | "resolution": "---", |
| "changed_after": "-14d", | | "email1": "nobody@mozilla.org", "email1_type": "not_equals", "email1_assigned_to": "1", |
| | "changed_after": "-1m", |
| "changed_before": "Now", | | "changed_before": "Now", |
| "changed_field": "resolution",
| | "include_fields": "id,priority,component,summary,assigned_to", |
| "changed_field_to": "FIXED",
| | "order": "priority,assigned_to" |
| "include_fields": "id,component,summary,assigned_to", | |
| "order": "assigned_to,id" | |
| } | | } |
| </bugzilla> | | </bugzilla> |
|
| |
| == Archive ==
| |
| * Original design notes:
| |
| ** [[Auto-tools/Projects/Treeherder/Current_TBPL_Feature_List]]
| |
| ** [[Auto-tools/Projects/Treeherder/Design_Draft]] (quite out of date now)
| |
| * [https://www.pivotaltracker.com/projects/749519 Pivotal Tracker project] (Deprecated)
| |
| * Etherpads: [https://etherpad.mozilla.org/sheriffing-with-treeherder sheriff-feedback-april] / [https://sheriffs.etherpad.mozilla.org/move-to-treeherder-requirements sheriff-feedback-june] / [https://etherpad.mozilla.org/treeherder-transition sheriff-transition-meeting-notes]
| |
| * Treeherder debrief Nov 2014 [https://etherpad.mozilla.org/treeherder-debrief treeherder-debrief]
| |