Confirmed users
3,816
edits
No edit summary |
No edit summary |
||
Line 1: | Line 1: | ||
* '''Ensure B2G has all test automation available to prepare for launch''' | * '''Ensure B2G has all test automation available to prepare for launch''' | ||
** {{ok|Deploy B2G Test Automation into TBPL running on Emulator and Panda platforms.}} | ** {{ok|Deploy B2G Test Automation into TBPL running on Emulator and Panda platforms.}} | ||
*** {{ok|Mochitest-plain (emulator first, panda second)}} | |||
*** {{ok|Mochitest-chome (emulator first, panda second)}} | |||
*** {{ok|Reftest (emulator first, panda second}} | |||
*** {{ok|xpcshell (panda)}} | |||
*** {{ok|webAPI (emulator only}} | |||
*** {{ok|webQA - gaia front end tests emulator and panda}} | |||
*** {{ok|gaia dev integration/unit tests emulator and panda}} | |||
** {{ok|Maintain B2G dog food update server as well as builds for development and QA while working to hand them off to releng}} | ** {{ok|Maintain B2G dog food update server as well as builds for development and QA while working to hand them off to releng}} | ||
** {{ok|Ensure all webAPIs testable through the emulator have comprehensive automated tests}} | ** {{ok|Ensure all webAPIs testable through the emulator have comprehensive automated tests}} | ||
** {{ok|Continue to aid QA in creating automated tests for B2G}} | ** {{ok|Continue to aid QA in creating automated tests for B2G}} | ||
* '''Expand and stabilize Test Automation on new and existing device platforms for NEON and ARMv6 support.''' | * '''Expand and stabilize Test Automation on new and existing device platforms for NEON and ARMv6 support.''' | ||
** {{ok|Have APC.io boards | ** {{ok|Have APC.io boards smoketest ready - rooted, working, plugged into Autophone}} | ||
** {{ok|Monitor deployment of large scale android and B2G panda board testing, aid with bug investigations}} | ** {{ok|Monitor deployment of large scale android and B2G panda board testing, aid with bug investigations}} | ||
** {{ok|Implement a new pool-based approach for device management in automation to increase stability on all device testing}} | ** {{ok|Implement a new pool-based approach for device management in automation to increase stability on all device testing}} | ||
** {{ok|Deploy Autophone | ** {{ok|Deploy Autophone mochitest and robocop test abilities. | ||
* '''Create and deploy performance automation for desktop, fennec, and B2G product benchmarks''' | * '''Create and deploy performance automation for desktop, fennec, and B2G product benchmarks''' | ||
** {{ok|Create Eideticker tests for B2G on panda boards and run them once a day}} | ** {{ok|Create Eideticker tests for B2G on panda boards and run them once a day}} | ||
** {{ok|Monitor Datazilla webservices and ensure our accuracy in detecting performance regressions on a per-push basis}} | ** {{ok|Monitor Datazilla webservices and ensure our accuracy in detecting performance regressions on a per-push basis}} | ||
** {{ok|Deploy Datazilla per-push UI to make it easier for developers to dig into the causes of performance regressions}} | ** {{ok|Deploy Datazilla per-push UI to make it easier for developers to dig into the causes of performance regressions}} | ||
* '''Update our bugzilla instance to 4.2 providing many benefits from UI to helping people keep track of bugs/projects from within Bugzilla.''' | * '''Update our bugzilla instance to 4.2 providing many benefits from UI to helping people keep track of bugs/projects from within Bugzilla.''' | ||
** {{ok|Deploy 4.2 version of Bugzilla}} | |||
** {{ok|Ensure that bug migration tools are in place for B2G so that we can share bugs with outside services and partners. | |||
** {{ok|Complete the 'Sandstone' CSS update}} | ** {{ok|Complete the 'Sandstone' CSS update}} | ||
** {{ok|Rework browserID <--> BMO integration}} | ** {{ok|Rework browserID <--> BMO integration}} | ||
** {{ok|Deliver user and product dashboards}} | ** {{ok|Deliver user and product dashboards}} | ||
** {{ok|Deploy tracking flags extension database optimization}} | ** {{ok|Deploy tracking flags extension database optimization}} | ||
* '''Make the process of sheriffing easier by updating TBPL and helping make it easier to handle the intermittent test problem''' | * '''Make the process of sheriffing easier by updating TBPL and helping make it easier to handle the intermittent test problem''' | ||
** | ** {{ok|Aid with B2G changes so that B2G oranges tracked in Orange factor, B2g tests show up in TBPL}} | ||
** | ** {{ok|Make it easier to sheriff and diagnose issues through better notification and automatic suggestions of issues for intermittent failures}} | ||
** {{ok|Split mochitest browser-chrome from mochitest-other to improve parallelism and overall turnaround time.}} | |||
** {{ok|Make orange factor update more than once a day to help make it easier to see changes over time}} | |||
** {{ok|Assist with getting Datazilla's UI integrated with TBPL, phase 1}} | |||
* '''Expand and Modernize the QA Automation framework as well as make it easier and more clear how to write automated tests for QA.''' | * '''Expand and Modernize the QA Automation framework as well as make it easier and more clear how to write automated tests for QA.''' | ||
** {{ok|Re-organize and improve the automated testing documentation at MDN}} | ** {{ok|Re-organize and improve the automated testing documentation at MDN}} | ||
** {{ok|Develop tests for WebRTC | ** {{ok|Develop first round of unit tests for WebRTC}} | ||
** {{ok|Ensure all B2G device WebAPIs testable through emulators have comprehensive low level test coverage | |||
''' Projects Defered due to B2G''' | |||
** {{defer|Port Eideticker to win32 and run tests there once a day to measure win32 snappiness for desktop}} | |||
** {{defer|Create pageload benchmark with networking shaper tests so that we can adequately track user-perceived performance across all devices and network types}} | |||
** {{defer|Complete REST API integration with Bugzilla}} | |||
** {{ok|Update selenium grid to use ESX on-demand to improve web QA capacity - start date pushed back due to B2G}} | |||
** {{ok|Finish and deploy mozmill 2.0 changes to frameworks and existing QA tests - start date pushed back due to B2G}} | |||
** Autophone goal scaled back and started later | |||
** Talos work scaled back as well. |