Confirmed users
9,511
edits
QA/Goals/2015q1: Difference between revisions
→Web QA
(→DOM) |
(→Web QA) |
||
| (18 intermediate revisions by 7 users not shown) | |||
| Line 56: | Line 56: | ||
== General Quality == | == General Quality == | ||
* {{ | * {{defer|<small>[whimboo, with help: ctalbert, kairo, releng]</small> Automate [https://github.com/mozilla/mozmill-ci/issues/535 kicking off update tests] so that the release process is automated end to end except manual spot checks}} | ||
* {{ | ** Defered due to time being spent on Mozmill -> Marionette Conversion and the possibility of doing update tests in buildbot. Will be coordinating with Releng | ||
** Training | * {{done|<small>[stephend, joint with a*team]</small> Coordinate training and prioritize efforts to convert P1 tests from Mozmill to the Marionette "[https://github.com/mozilla/firefox-ui-tests Green-light]" test framework in preparation for E10s}} | ||
** Training video was posted to Air Mozilla: https://air.mozilla.org/mozmill-to-marionette-conversion-bootcamp-training/ | |||
** Remaining conversion work in dependencies from this tracking bug: [https://bugzilla.mozilla.org/show_bug.cgi?id=1129843 bug 1129843] | ** Remaining conversion work in dependencies from this tracking bug: [https://bugzilla.mozilla.org/show_bug.cgi?id=1129843 bug 1129843] | ||
* {{ok|<small>[whimboo, a*team]</small> Get Marionette Green-light tests running in the QA CI system for at least Nightly tests}} | * {{ok|<small>[whimboo, a*team]</small> Get Marionette Green-light tests running in the QA CI system for at least Nightly tests}} | ||
| Line 75: | Line 75: | ||
=== DOM === | === DOM === | ||
* {{ | * {{done|<small>[ashughes]</small> - Document best practices for triage and test writing for DOM issues to pave the way for community involvement}} | ||
** work is ongoing but the v1 doc can be found now on [https://developer.mozilla.org/en-US/docs/Mozilla/QA/Helping_the_DOM_team developer.mozilla.org] | |||
* {{done|<small>[ashughes]</small> - Establish baseline metrics for specification coverage, test stability, and other DOM quality metrics and track them throughout Q1}} | * {{done|<small>[ashughes]</small> - Establish baseline metrics for specification coverage, test stability, and other DOM quality metrics and track them throughout Q1}} | ||
* {{ | ** work is ongoing but the v1 doc can be found now on [[QA/Platform/DOM/Metrics|wiki.mozilla.org]] | ||
** | * {{risk|<small>[ashughes]</small> - Create several new automated tests/get existing Web-platform tests working to support the Picture tag and ServiceWorkers specifications}} | ||
** this is at risk due to re-org and higher priority deliverables being added (eg. Betabreakers) | |||
=== Security/Privacy === | === Security/Privacy === | ||
| Line 94: | Line 96: | ||
* {{done|<small>[mbrandt and shared with MDN dev]</small> Develop and stand up a JavaScript-based testing infrastructure (proof-of-concept)}} which allows Web development and Web QA to be maximally effective by sharing key points of test infrastructure, visibility, and process/workflow: | * {{done|<small>[mbrandt and shared with MDN dev]</small> Develop and stand up a JavaScript-based testing infrastructure (proof-of-concept)}} which allows Web development and Web QA to be maximally effective by sharing key points of test infrastructure, visibility, and process/workflow: | ||
** {{done|<small>[mbrandt]</small> at the end of the quarter, do a writeup (blog post?) covering the progress and any next steps for Q1 2015}} | ** {{done|<small>[mbrandt]</small> at the end of the quarter, do a writeup (blog post?) covering the progress and any next steps for Q1 2015}} | ||
* {{ | * {{done|<small>[bsilverberg]</small> - QA Metrics Dashboard - collate metrics on a page of links as new metrics are proven to be useful}} | ||
** this is to support, and have a dependency on, Kairo's goal under "General Quality," above | ** this is to support, and have a dependency on, Kairo's goal under "General Quality," above | ||
* {{ | *** Added http://arewestableyet.com/graph/ to http://bobsilverberg.github.io/qa-dashboard/ | ||
** | * {{done|<small>[davehunt]</small> - bug 1112555 - Make Web QA's Jenkins public}} | ||
** https://blog.mozilla.org/webqa/2015/03/05/web-qa-publishes-automated-test-results/ | |||
* {{ok|<small>[mbrandt/bsilverberg] - </small>Identify and deliver on the highest [https://www.pivotaltracker.com/n/projects/280483 priority/biggest pain-points of MozTrap after the API work (API to be covered by camd)]}} | * {{ok|<small>[mbrandt/bsilverberg] - </small>Identify and deliver on the highest [https://www.pivotaltracker.com/n/projects/280483 priority/biggest pain-points of MozTrap after the API work (API to be covered by camd)]}} | ||
* {{ | * {{done|<small>[krupa]</small> - Have automation coverage for Payments including in-app payments}} | ||
** total scope of work (done vs. remaining) here: [https://bugzilla.mozilla.org/showdependencytree.cgi?id=1015251&hide_resolved=0 bug 1015251] | ** total scope of work (done vs. remaining) here: [https://bugzilla.mozilla.org/showdependencytree.cgi?id=1015251&hide_resolved=0 bug 1015251] | ||
* {{ | ** In-app Payments tests landed in https://github.com/mozilla/marketplace-tests-gaia/commit/e39b0c2a0ac3061d9d511ba9010e09ceece8ac31 | ||
* {{done|<small>[krupa]</small> - Coordinate and drive quality efforts to prepare for AMO to require add-on signing}} | |||
** Still on-track; landing in Fx39 Beta, in mid-May | |||
== Platform QA == | == Platform QA == | ||
* {{ok|<small>[Marcia, Syd]</small> Eliminate Flash issues on YouTube by supporting the media team in shipping MSE on YouTube for Windows Vista and higher through manual and automated testing}} | * {{ok|<small>[Marcia, Syd]</small> Eliminate Flash issues on YouTube by supporting the media team in shipping MSE on YouTube for Windows Vista and higher through manual and automated testing}} | ||
=== Prioritized Tasks === | |||
* {{done|<small>[Maja]</small> Create and deploy a suite of automated YouTube playback tests. }} | |||
* {{done|<small>[Maja]</small> Write a tool to pull a subset of YouTube URLs from crash-stats and feed that to the automated playback tests. }} | |||
== Community == | == Community == | ||
* [marcia] Build community effort to support the Buddy Up project | * {{done|<small>[marcia]</small> Build community effort to support the Buddy Up project }} | ||
* [ashughes] Help create at least three small, focused test-day events that focus on helping contributors be more effective | * {{ok|<small>[ashughes]</small> Help create at least three small, focused test-day events that focus on helping contributors be more effective}} | ||
** we have three events planned this quarter: Bangladesh, Argentina/Spain, and the UK | |||