Confirmed users
1,905
edits
Auto-tools/Goals/2014Q4: Difference between revisions
→{{done|}} Prototype alternate bug views [glob]: show goal as missed
(→Developer Productivity: Autoland done) |
(→{{done|}} Prototype alternate bug views [glob]: show goal as missed) |
||
| (12 intermediate revisions by 5 users not shown) | |||
| Line 7: | Line 7: | ||
== Firefox == | == Firefox == | ||
=== Multi-binary devtools harness [ted] === | === {{done|}} Multi-binary devtools harness [ted] === | ||
Prototype a test harness which enables developers to run tests involving Firefox and another binary, such as Chrome, Safari, or a Firefox OS Device. | Prototype a test harness which enables developers to run tests involving Firefox and another binary, such as Chrome, Safari, or a Firefox OS Device. | ||
| Line 14: | Line 14: | ||
* Project page: TBD | * Project page: TBD | ||
=== Mozmill e10s [jgriffin, dburns, ahal, chmanchester] === | === {{done|}} Mozmill e10s [jgriffin, dburns, ahal, chmanchester] === | ||
QA runs a valuable set of release tests on Firefox using Mozmill, which is not e10s compatible. To address this, we're going to enable the conversion of mozmill tests to Marionette, which will support e10s, and which has better ongoing support. | QA runs a valuable set of release tests on Firefox using Mozmill, which is not e10s compatible. To address this, we're going to enable the conversion of mozmill tests to Marionette, which will support e10s, and which has better ongoing support. | ||
| Line 26: | Line 26: | ||
Deliver a functional prototype of the Autoland service that can be used for opt-in dogfooding. Full release to the entire community will occur after load testing and deployment in 2015Q1. | Deliver a functional prototype of the Autoland service that can be used for opt-in dogfooding. Full release to the entire community will occur after load testing and deployment in 2015Q1. | ||
* Slight change to scope: we have a proof of concept that has been used to land to | * Slight change to scope: we have a proof of concept that has been used to land to a clone of mozilla-central running on the Transplant service. Given the plans for Autoland in 2015 (interfacing with MozReview), there's no point in further work on this particular prototype. | ||
* why: Developer productivity gains across the board, since this would reduce the amount of time required by developers to land a patch. | * why: Developer productivity gains across the board, since this would reduce the amount of time required by developers to land a patch. | ||
* {{bug|1043010}} | * {{bug|1043010}} | ||
| Line 48: | Line 48: | ||
== Performance == | == Performance == | ||
=== Move dzAlerts from Datazilla to Treeherder [ekyle] === | === {{missed|}} Move dzAlerts from Datazilla to Treeherder [ekyle] === | ||
Make dzAlerts (which produces Talos performance regression alerts) ingest data from Treeherder, rather than Datazilla, as the latter doesn't scale well with Talos data. | Make dzAlerts (which produces Talos performance regression alerts) ingest data from Treeherder, rather than Datazilla, as the latter doesn't scale well with Talos data. | ||
| Line 54: | Line 54: | ||
* {{bug|1092206}} | * {{bug|1092206}} | ||
* Project page: https://wiki.mozilla.org/Auto-tools/Projects/Alerts | * Project page: https://wiki.mozilla.org/Auto-tools/Projects/Alerts | ||
* missed because Treeherder wasn't ingesting all of the Talos performance data, a fact which wasn't obvious until it was too late to address it in time to complete this goal | |||
=== {{defer|}} Create tests for the "charts" (dashboard) service [ekyle] === | === {{defer|}} Create tests for the "charts" (dashboard) service [ekyle] === | ||
| Line 63: | Line 64: | ||
* Project page: https://wiki.mozilla.org/Auto-tools/Projects/Charts | * Project page: https://wiki.mozilla.org/Auto-tools/Projects/Charts | ||
=== Make Talos compatible with e10s [wlach/jmaher] === | === {{done|Make Talos compatible with e10s [wlach/jmaher]}} === | ||
Adapt Talos to work with Firefox in e10s mode, and get Talos tests running in e10s mode scheduled on trunk branches. | Adapt Talos to work with Firefox in e10s mode, and get Talos tests running in e10s mode scheduled on trunk branches. | ||
| Line 77: | Line 78: | ||
* Project pages: https://wiki.mozilla.org/Electrolysis, https://wiki.mozilla.org/Buildbot/Talos | * Project pages: https://wiki.mozilla.org/Electrolysis, https://wiki.mozilla.org/Buildbot/Talos | ||
=== Monitor and file bugs for Talos performance regressions [jmaher] === | === {{done|Monitor and file bugs for Talos performance regressions [jmaher]}} === | ||
We will monitor the [http://54.215.155.53:8080/alerts.html# Alert Manager] for regressions in Talos tests and file bugs as needed. Sometime post-Q4 we will roll this into the normal sheriffing workflow, after we have the tooling to support that. | We will monitor the [http://54.215.155.53:8080/alerts.html# Alert Manager] for regressions in Talos tests and file bugs as needed. Sometime post-Q4 we will roll this into the normal sheriffing workflow, after we have the tooling to support that. | ||
| Line 86: | Line 87: | ||
== Treeherder == | == Treeherder == | ||
=== Sheriff and developer improvements [jeads, mdoglio, camd, edmorley] === | === {{done|}} Sheriff and developer improvements [jeads, mdoglio, camd, edmorley] === | ||
Continue to improve sheriff and developer workflows in Treeherder so we can effectively obsolete TBPL in the future. | Continue to improve sheriff and developer workflows in Treeherder so we can effectively obsolete TBPL in the future. | ||
| Line 102: | Line 103: | ||
* Project page: https://mana.mozilla.org/wiki/display/IPPP/GMAIL+Project+Page | * Project page: https://mana.mozilla.org/wiki/display/IPPP/GMAIL+Project+Page | ||
=== Prototype alternate bug views [glob] === | === {{miss|}} Prototype alternate bug views [glob] === | ||
Prototype some alternate bug views which can both decrease page load times and improve the user experience for specific use cases. | Prototype some alternate bug views which can both decrease page load times and improve the user experience for specific use cases. | ||
| Line 111: | Line 112: | ||
== General Automation == | == General Automation == | ||
=== mochitest-chrome for B2G [gbrown] === | === {{done|}} mochitest-chrome for B2G [gbrown] === | ||
Developers have requested that we add support for running the mochitest-chrome harness on FirefoxOS, so that they can write chrome-related tests without using hacky and problematic SpecialPowers workarounds in mochitest-plain. This goal does not include porting existing mochitest-chrome tests to B2G, just allowing new tests to be written using this harness. | Developers have requested that we add support for running the mochitest-chrome harness on FirefoxOS, so that they can write chrome-related tests without using hacky and problematic SpecialPowers workarounds in mochitest-plain. This goal does not include porting existing mochitest-chrome tests to B2G, just allowing new tests to be written using this harness. | ||
| Line 118: | Line 119: | ||
* Project page: N/A | * Project page: N/A | ||
=== Scope the consolidation of non-buildbot tests [bc] === | === {{miss|}} Scope the consolidation of non-buildbot tests [bc] === | ||
We have several separate test suites that run on independent systems, like Autophone, Eideticker, mozbench, power profiling tests, etc. Each of these requires separate maintenance and feature development. We will scope the effort involved in consolidating different aspects of these, including test scheduling, test reporting, Android and Firefox OS device maintenance, and the use of buildbot for desktop tests, so we can determine whether or not we should attempt such a consolidation in 2015. | We have several separate test suites that run on independent systems, like Autophone, Eideticker, mozbench, power profiling tests, etc. Each of these requires separate maintenance and feature development. We will scope the effort involved in consolidating different aspects of these, including test scheduling, test reporting, Android and Firefox OS device maintenance, and the use of buildbot for desktop tests, so we can determine whether or not we should attempt such a consolidation in 2015. | ||
| Line 124: | Line 125: | ||
* bug: N/A | * bug: N/A | ||
* Project page: will be created as a deliverable for this work. | * Project page: will be created as a deliverable for this work. | ||
* missed because of work done (by request) to get video tests running in Autophone | |||
=== Move more mozharness config info into the tree [armenzg] === | === {{done|}} Make it possible to test mozharness changes on try [armenzg] === | ||
We want to be able to test mozharness changes on try, without having to run them on ash or cedar. | |||
* why: faster development of mozharness-related code | |||
* bug: {{bug|791924}} | |||
* Project page: https://wiki.mozilla.org/User:Armenzg/Proposals/Mozharness_changes | |||
=== {{defer|}} Move more mozharness config info into the tree [armenzg] === | |||
note: deferred in favor of above goal of supporting mozharness-try | |||
We want to move more config details that mozharness uses from the mozharness repo into the tree, e.g. run_file_names, specific_tests_zip_dirs, in_tree_config, all_$category_suites. The exact details moved will vary by script, but in general we'd like to move all the suite-specific details into the tree and out of mozharness. | We want to move more config details that mozharness uses from the mozharness repo into the tree, e.g. run_file_names, specific_tests_zip_dirs, in_tree_config, all_$category_suites. The exact details moved will vary by script, but in general we'd like to move all the suite-specific details into the tree and out of mozharness. | ||
| Line 134: | Line 145: | ||
== Community == | == Community == | ||
=== A-Team Boot Camp [wlach] === | === {{done|}} A-Team Boot Camp [wlach] === | ||
Create a set of documentation to help onboard new contributors and others to working with the automation that the A-Team maintains. | Create a set of documentation to help onboard new contributors and others to working with the automation that the A-Team maintains. | ||
* why: Make the A-Team more accessible to community members, and reduce the time we spend mentoring contributors on basic tasks. | * why: Make the A-Team more accessible to community members, and reduce the time we spend mentoring contributors on basic tasks. | ||
* bug: N/A | * bug: N/A | ||
* | * initial version available: http://ateam-bootcamp.readthedocs.org/en/latest/index.html | ||
=== Make our projects contributor-friendly [jmaher, jgriffin, all] === | === {{done|Make our projects contributor-friendly [jmaher, jgriffin, all]}} === | ||
By the end of the quarter, all projects which are continuing to Q1 should be "contributor-friendly"...with a Project Page, up-to-date docs, etc. | By the end of the quarter, all projects which are continuing to Q1 should be "contributor-friendly"...with a Project Page, up-to-date docs, etc. | ||