Auto-tools/Goals/2012Q4: Difference between revisions

no edit summary
No edit summary
No edit summary
Line 1: Line 1:
* '''Ensure B2G has all test automation available to prepare for launch'''
* '''Ensure B2G has all test automation available to prepare for launch'''
** {{ok|Deploy B2G Test Automation into TBPL running on Emulator and Panda platforms.}}
** {{ok|Deploy B2G Test Automation into TBPL running on Emulator and Panda platforms.}}
*** {{ok|Mochitest-plain (emulator first, panda second)}}
*** {{ok|Mochitest-chome (emulator first, panda second)}}
*** {{ok|Reftest (emulator first, panda second}}
*** {{ok|xpcshell (panda)}}
*** {{ok|webAPI (emulator only}}
*** {{ok|webQA - gaia front end tests emulator and panda}}
*** {{ok|gaia dev integration/unit tests emulator and panda}}
** {{ok|Maintain B2G dog food update server as well as builds for development and QA while working to hand them off to releng}}
** {{ok|Maintain B2G dog food update server as well as builds for development and QA while working to hand them off to releng}}
** {{ok|Ensure all webAPIs testable through the emulator have comprehensive automated tests}}
** {{ok|Ensure all webAPIs testable through the emulator have comprehensive automated tests}}
** {{ok|Continue to aid QA in creating automated tests for B2G}}
** {{ok|Continue to aid QA in creating automated tests for B2G}}
* '''Expand and stabilize Test Automation on new and existing device platforms for NEON and ARMv6 support.'''
* '''Expand and stabilize Test Automation on new and existing device platforms for NEON and ARMv6 support.'''
** {{ok|Have APC.io boards chassis-ready -- rooted, smoketested, and working for ARMv6 testing}}
** {{ok|Have APC.io boards smoketest ready - rooted, working, plugged into Autophone}}
** {{ok|Monitor deployment of large scale android and B2G panda board testing, aid with bug investigations}}
** {{ok|Monitor deployment of large scale android and B2G panda board testing, aid with bug investigations}}
** {{ok|Implement a new pool-based approach for device management in automation to increase stability on all device testing}}
** {{ok|Implement a new pool-based approach for device management in automation to increase stability on all device testing}}
** {{ok|Deploy Autophone with 40 phones and 70% up time}}
** {{ok|Deploy Autophone mochitest and robocop test abilities.
* '''Create and deploy performance automation for desktop, fennec, and B2G product benchmarks'''
* '''Create and deploy performance automation for desktop, fennec, and B2G product benchmarks'''
** {{ok|Create Eideticker tests for B2G on panda boards and run them once a day}}
** {{ok|Create Eideticker tests for B2G on panda boards and run them once a day}}
** {{ok|Port Eideticker to win32 and run tests there once a day to measure win32 snappiness for desktop}}
** {{ok|Create pageload benchmark with networking shaper tests so that we can adequately track user-perceived performance across all devices and network types}}
** {{ok|Monitor Datazilla webservices and ensure our accuracy in detecting performance regressions on a per-push basis}}
** {{ok|Monitor Datazilla webservices and ensure our accuracy in detecting performance regressions on a per-push basis}}
** {{ok|Deploy Datazilla per-push UI to make it easier for developers to dig into the causes of performance regressions}}
** {{ok|Deploy Datazilla per-push UI to make it easier for developers to dig into the causes of performance regressions}}
* '''Update our bugzilla instance to 4.2 providing many benefits from UI to helping people keep track of bugs/projects from within Bugzilla.'''
* '''Update our bugzilla instance to 4.2 providing many benefits from UI to helping people keep track of bugs/projects from within Bugzilla.'''
** {{ok|Deploy 4.2 version of Bugzilla}}
** {{ok|Ensure that bug migration tools are in place for B2G so that we can share bugs with outside services and partners.
** {{ok|Complete the 'Sandstone' CSS update}}
** {{ok|Complete the 'Sandstone' CSS update}}
** {{ok|Complete the REST API integration}}
** {{ok|Rework browserID <--> BMO integration}}
** {{ok|Rework browserID <--> BMO integration}}
** {{ok|Deliver user and product dashboards}}
** {{ok|Deliver user and product dashboards}}
** {{ok|Deploy tracking flags extension database optimization}}
** {{ok|Deploy tracking flags extension database optimization}}
* '''Make the process of sheriffing easier by updating TBPL and helping make it easier to handle the intermittent test problem'''
* '''Make the process of sheriffing easier by updating TBPL and helping make it easier to handle the intermittent test problem'''
** TBD
** {{ok|Aid with B2G changes so that B2G oranges tracked in Orange factor, B2g tests show up in TBPL}}
** TBD
** {{ok|Make it easier to sheriff and diagnose issues through better notification and automatic suggestions of issues for intermittent failures}}
** {{ok|Split mochitest browser-chrome from mochitest-other to improve parallelism and overall turnaround time.}}
** {{ok|Make orange factor update more than once a day to help make it easier to see changes over time}}
** {{ok|Assist with getting Datazilla's UI integrated with TBPL, phase 1}}
* '''Expand and Modernize the QA Automation framework as well as make it easier and more clear how to write automated tests for QA.'''
* '''Expand and Modernize the QA Automation framework as well as make it easier and more clear how to write automated tests for QA.'''
** {{ok|Finish and deploy mozmill 2.0 changes to frameworks and existing QA tests}}
** {{ok|Update selenium grid to use ESX on-demand to improve web QA capacity}}
** {{ok|Re-organize and improve the automated testing documentation at MDN}}
** {{ok|Re-organize and improve the automated testing documentation at MDN}}
** {{ok|Develop tests for WebRTC, B2G and other high profile projects}}
** {{ok|Develop first round of unit tests for WebRTC}}
** {{ok|Ensure all B2G device WebAPIs testable through emulators have comprehensive low level test coverage
 
''' Projects Defered due to B2G'''
** {{defer|Port Eideticker to win32 and run tests there once a day to measure win32 snappiness for desktop}}
** {{defer|Create pageload benchmark with networking shaper tests so that we can adequately track user-perceived performance across all devices and network types}}
** {{defer|Complete REST API integration with Bugzilla}}
** {{ok|Update selenium grid to use ESX on-demand to improve web QA capacity - start date pushed back due to B2G}}
** {{ok|Finish and deploy mozmill 2.0 changes to frameworks and existing QA tests - start date pushed back due to B2G}}
** Autophone goal scaled back and started later
** Talos work scaled back as well.
Confirmed users
3,816

edits