Confirmed users
3,816
edits
QA/Goals/2015q1: Difference between revisions
Jump to navigation
Jump to search
no edit summary
m (really fixed format this time) |
No edit summary |
||
| Line 114: | Line 114: | ||
== General Quality == | == General Quality == | ||
* {{ok|<small>[whimboo, with help: ctalbert, kairo, releng]</small> Automate [https://github.com/mozilla/mozmill-ci/issues/535 kicking off update tests] so that the release process is automated end to end except manual spot checks}} | |||
* {{ok|<small>[stephend, joint with a*team]</small> Coordinate training and prioritize efforts to convert P1 tests from Mozmill to the Marionette "Green-light" test framework in preparation for E10s}} | |||
* {{ok|<small>[whimboo, a*team]</small> Get Marionette Green-light tests running in the QA CI system for at least Nightly tests}} | |||
* {{ok|<small>[mschifer]</small> Create and monitor a set of risk factor metrics to understand risks in areas where we have no embedded QA}} | |||
* {{ok|<small>[kairo]</small> Create a better concept for tracking crash-stats that more reflects the actual risk and severity of the crashes involved (than the current top 10 list)}} | |||
== Firefox == | == Firefox == | ||
* {{ | === E10s === | ||
* {{ok|<small>[lizzard, juanb]</small> - Establish baselines for automated test coverage, crash-stats, add-on compatibility, and memory footprint for e10s Nightly and track them throughout Q1.}} | |||
* {{ok|<small>[lizzard, juanb]</small> - Improve the automated test coverage for E10s enabled builds by turning on tests for a new platform and ensuring bugs filed for perma-fails.}} | |||
* {{ok|<small>[lizzard, juanb]</small> - Engage community to help improve addon compatibility testing for e10s}} | |||
** milestones/deliverables: https://etherpad.mozilla.org/e10s-strategy-qa | ** milestones/deliverables: https://etherpad.mozilla.org/e10s-strategy-qa | ||
=== DOM === | |||
* {{ok|<small>[tracy]</small> - Document best practices for triage and test writing for DOM issues to pave the way for community involvement}} | |||
* {{ok|<small>[ashughes]</small> - Establish baseline metrics for specification coverage, test stability, and other DOM quality metrics and track them throughout Q1}} | |||
* {{ok|<small>[tracy, ashughes]</small> - Create several new automated tests/get existing web platform tests working to support the Picture tag and ServiceWorkers specifications}} | |||
** milestones/deliverables: https://etherpad.mozilla.org/qa-dom-strategy | |||
=== Security/Privacy === | |||
* {{ok|<small>[mwobensmith]</small> - Set up AWS instance to run large TLS regression test}} | |||
* {{ok|<small>[mwobensmith]</small> - Create a fast running subset of the larger TLS test and automate to run per-release (i.e. pre-beta)}} | |||
* {{ok|<small>[kjoziwak]</small> - Monitor and create telemetry probes to gather data to help serve as an early warning system for risky changes}} | |||
=== Fennec === | |||
{{ok|<small>[aaronmt, kbrosnan] - Slim down the per release smoketest by at least 25%}} | |||
{{ok|<small>[kbrosnan] - Investigate using marionette/firefox greenlight on Fennec, create a plan and prioritized set of targets for the marionette team}} | |||
{{ok|<small>[aaronmt] - Fix or discontinue appthwak automation}} | |||
{{ok|<small> | |||
== Web QA == | == Web QA == | ||
* {{prev|<small>[mbrandt and shared with MDN dev]</small> Develop and stand up a JavaScript-based testing infrastructure (proof-of-concept)}} which allows Web development and Web QA to be maximally effective by sharing key points of test infrastructure, visibility, and process/workflow: | * {{prev|<small>[mbrandt and shared with MDN dev]</small> Develop and stand up a JavaScript-based testing infrastructure (proof-of-concept)}} which allows Web development and Web QA to be maximally effective by sharing key points of test infrastructure, visibility, and process/workflow: | ||
** {{prev|<small>[mbrandt]</small> at the end of the quarter, do a writeup (blog post?) covering the progress and any next steps for Q1 2015}} | ** {{prev|<small>[mbrandt]</small> at the end of the quarter, do a writeup (blog post?) covering the progress and any next steps for Q1 2015}} | ||
* {{ | * {{ok|<small>[bsilverberg?]</small> - QA Metrics Dashboard - collate metrics on a page of links as new metrics are proven to be useful}} | ||
* {{ | * {{ok|<small>[davehunt]</small> - bug 1112555 Make Web QA's Jenkins public}} | ||
* {{ | * {{ok|<small>[mbrandt/bsilverberg] - </small>identify and deliver on the highest [https://www.pivotaltracker.com/n/projects/280483 priority/biggest pain-points of MozTrap]}}: | ||
* {{prev|<small>[krupa]</small> - Have automation coverage for Payments including in-app payments}} | |||
** total scope of work (done vs. remaining) here: [https://bugzilla.mozilla.org/showdependencytree.cgi?id=1015251&hide_resolved=0 bug 1015251] | |||
* {{ok|<small>[krupa]</small> - Coordinate and drive quality efforts to prepare for AMO to require addon signing}} | |||
== Platform QA == | == Platform QA == | ||
* {{ok|<small>[Marcia, Syd]</small> Eliminate Flash issues on youtube by supporting the media team in shipping MSE on youtube for windows vista and higher through manual and automated testing}} | |||
== Community == | == Community == | ||
* [marcia] Build community effort to support the Buddy Up project | * [marcia] Build community effort to support the Buddy Up project | ||
* [ashughes] Help create at least three small, focused test-day events that focus on helping contributors be more effective | |||