Confirmed users
1,927
edits
Auto-tools/Goals/2013Q3: Difference between revisions
m
→P0: Bugzilla
m (→P0: Bugzilla) |
|||
| (18 intermediate revisions by 5 users not shown) | |||
| Line 13: | Line 13: | ||
** {{done|Built in-cloud on AWS}} | ** {{done|Built in-cloud on AWS}} | ||
** {{defer|Integration with Treeherder}} | ** {{defer|Integration with Treeherder}} | ||
** {{ | ** {{done|Allows tests to be defined simply in the main m-c tree}} | ||
** {{ | ** {{done|Runs WebRTC tests}} | ||
** {{defer|Could also use this system to do a sanity check on the social API partner integration once the marionette based tests for social API are written}} | ** {{defer|Could also use this system to do a sanity check on the social API partner integration once the marionette based tests for social API are written}} | ||
** {{ | ** {{done|Should use an off-the-shelf scheduler - either jenkins or buildbot or something.}} | ||
=== P0: B2G General === | === P0: B2G General === | ||
* {{done|Make B2G tests in TBPL more sheriffable, by providing a tool in TBPL (Treeherder?) to compare revisions of all B2G repos between arbitrary builds}} | * {{done|Make B2G tests in TBPL more sheriffable, by providing a tool in TBPL (Treeherder?) to compare revisions of all B2G repos between arbitrary builds}} | ||
* {{done|Finish getting gaia-unit-tests and gaia-ui-tests running on all trees in TBPL}} | * {{done|Finish getting gaia-unit-tests and gaia-ui-tests running on all trees in TBPL}} | ||
* {{ | * {{miss|(Depends on James Lal's dev team): Deploy gaia-integration tests into TBPL}} | ||
** ''NOTE: We have an npm mirror; need to write mozharness script'' | ** ''NOTE: We have an npm mirror; need to write mozharness script'' | ||
* {{ | * {{miss|Stand up tests on new Android 4.2 emulator (this is potentially going to be very time consuming)}} | ||
** ''NOTE: Waiting for rel-eng to schedule 4.2 builds/tests on cedar'' | ** ''NOTE: Waiting for rel-eng to schedule 4.2 builds/tests on cedar'' | ||
* {{done|Get b2g jsreftests working, so we can turn off no-ion builds/tests (stretch goal)}} | * {{done|Get b2g jsreftests working, so we can turn off no-ion builds/tests (stretch goal)}} | ||
| Line 33: | Line 33: | ||
=== P0: Eideticker === | === P0: Eideticker === | ||
* {{ | * {{done|Create responsiveness test for firefox on Android test using Eideticker}} | ||
* {{ | * {{done|Create a set (at least 2) of tests for b2g Eideticker to measure user perceived application startup and scrolling performance, and run on a daily basis}} | ||
=== P1: Marionette === | === P1: Marionette === | ||
| Line 64: | Line 64: | ||
** {{done|add additional alerts for disk flushes, too many disk accesses with talos xperf}} | ** {{done|add additional alerts for disk flushes, too many disk accesses with talos xperf}} | ||
** {{done|continue modifying existing tests to make the data more useful, improve documentation on what all tests measure}} | ** {{done|continue modifying existing tests to make the data more useful, improve documentation on what all tests measure}} | ||
** {{ | ** {{done|modify talos ts_dirty tests to have updated profiles and measure useful scenarios}} | ||
** {{ | ** {{miss|have sheriff/developer dashboards for datazilla, statistical analysis of all test types with alerts for detected regressions at <2% noise}} | ||
* Other: | * Other: | ||
** {{done|continue working with developers to refine microperf}} | ** {{done|continue working with developers to refine microperf}} | ||
** {{ | ** {{done|deliver a prototype for real world benchmark using the live internet to aid the js team}} | ||
** {{defer|provide help to the servo/rust team as they start measuring performance}} | ** {{defer|provide help to the servo/rust team as they start measuring performance}} | ||
** {{ | ** {{done|investigate ways to involve community into the process of measurement and analysis}} | ||
=== P1: MozBase === | === P1: MozBase === | ||
* {{ | * {{done|unittests for all modules}} | ||
* {{ | * {{miss|m-c commit watcher to notify mozbase owners of unexpected commits}} | ||
* {{ | * {{miss|full mochitest and reftest integration with mozprofile/mozprocess/mozrunner (excluding Fennec)}} | ||
** ''NOTE: mozprofile work done; the remainder is just being started'' | ** ''NOTE: mozprofile work done; the remainder is just being started'' | ||
* {{done|continue to standardize + impove dev practices}} | * {{done|continue to standardize + impove dev practices}} | ||
* {{done|structured logging}} | * {{done|structured logging}} | ||
* {{ | * {{done|manifestdestiny API improvements and fixes}} | ||
* {{done|add mozprocess documentation}} | * {{done|add mozprocess documentation}} | ||
=== P2: mozmill 2.0 released === | === P2: mozmill 2.0 released === | ||
* {{ | * {{done|Release 2.0 (should be done Q2)}} | ||
* {{done|SV leads metro testing using mozmill}} | * {{done|SV leads metro testing using mozmill}} | ||
| Line 92: | Line 92: | ||
* {{done|Deploy native REST API}} | * {{done|Deploy native REST API}} | ||
* {{done|Optimize and expand native REST API as needed to support 3rd party UI building efforts}} | * {{done|Optimize and expand native REST API as needed to support 3rd party UI building efforts}} | ||
* {{ | * {{done|Stand up reviewboard as a semi-independent reviewing service}} | ||
* {{ | ** NOTE: Up at https://reviewboard.mozilla.org/, but waiting on proper cert and http -> https redirect from IT. | ||
* {{done|Tracking flag migration}} | |||
** Migration script is ready, but IT outage will not happen until early Q4. | ** Migration script is ready, but IT outage will not happen until early Q4. | ||
=== P0: TreeHerder === | === P0: TreeHerder === | ||
* {{ | * {{miss|Deliver UI parity with existing TBPL}} | ||
* {{done|Implement UI plugin architecture}} | * {{done|Implement UI plugin architecture}} | ||
* {{done|Set up publicly accessible instance}} | * {{done|Set up publicly accessible instance}} | ||
| Line 109: | Line 110: | ||
* {{done|Refactor datazilla to remove all portions of the old UI, unnecessary controllers, and legacy code not in production}} | * {{done|Refactor datazilla to remove all portions of the old UI, unnecessary controllers, and legacy code not in production}} | ||
* {{done|Provide datazilla landing page that displays data for all projects}} | * {{done|Provide datazilla landing page that displays data for all projects}} | ||
* {{ | * {{miss|Replace graph server for talos on m-c,m-i}} | ||
** Everything in place, except alerting which will carry into Q4 | |||
== Process == | == Process == | ||
=== P1: Infrastructure Load === | === P1: Infrastructure Load === | ||
* {{done|parallelize xpcshell test framework}} | * {{done|parallelize xpcshell test framework}} | ||
* {{ | * {{done|analyze always passing tests and determine a strategy for turning tests off selectively/running intermittently}} | ||
* {{ | * {{done|work with releng to turn on code coverage builds}} | ||
=== P1: Sheriff goals === | === P1: Sheriff goals === | ||
* {{ | * {{done|Provide documentation and how to steps to enable a community sheriff to be effective}} | ||
* {{ | * {{miss|Wire documentation into bugsAhoy/whatcanIdoForMozilla}} | ||
* {{done|Fix P1/low hanging fruit of the Sheriffing Tools (see full goals list)}} | * {{done|Fix P1/low hanging fruit of the Sheriffing Tools (see full goals list)}} | ||
* {{done|Drive treeherder UI functional specification}} | * {{done|Drive treeherder UI functional specification}} | ||
| Line 126: | Line 128: | ||
=== P1: Bugmaster === | === P1: Bugmaster === | ||
* {{done|Create a contribution pipeline for 30 incoming bugmasters and track them through each stage of involvement in order to get 1 to the state where they regularly do triage.}} | * {{done|Create a contribution pipeline for 30 incoming bugmasters and track them through each stage of involvement in order to get 1 to the state where they regularly do triage.}} | ||
* {{ | * {{done|Improve Bugzilla Help pages.}} | ||
* [https://wiki.mozilla.org/Auto-tools/Projects/Bugmaster_2013Q3 Full goals here] | * [https://wiki.mozilla.org/Auto-tools/Projects/Bugmaster_2013Q3 Full goals here] | ||