Auto-tools/Goals/2012Q4

From MozillaWiki
< Auto-tools‎ | Goals
Revision as of 11:06, 30 October 2012 by Edmorley (talk | contribs) (Update sheriffing goals)
Jump to navigation Jump to search
  • Ensure B2G has all test automation available to prepare for launch
    • [ON TRACK] Deploy B2G Test Automation into TBPL running on Emulator and Panda platforms. Specifically we are turning on:
      • [ON TRACK] Mochitest-plain (emulator first, panda second)
      • [ON TRACK] Mochitest-chome (emulator first, panda second)
      • [ON TRACK] Reftest (emulator first, panda second
      • [ON TRACK] xpcshell (panda)
      • [ON TRACK] webAPI (emulator only
      • [ON TRACK] webQA - gaia front end tests emulator and panda
      • [ON TRACK] gaia dev integration/unit tests emulator and panda
    • [ON TRACK] Maintain B2G dog food update server as well as builds for development and QA while working to hand them off to releng
    • [ON TRACK] Ensure all webAPIs testable through the emulator have comprehensive automated tests
    • [ON TRACK] Continue to aid QA in creating automated tests for B2G
  • Expand and stabilize Test Automation on new and existing device platforms for NEON and ARMv6 support.
    • [ON TRACK] Have APC.io (ARMv6)boards smoketest ready - rooted, working, plugged into Autophone
    • [ON TRACK] Monitor deployment of large scale android and B2G panda board testing, aid with bug investigations
    • [ON TRACK] Implement a new pool-based approach for device management in automation to increase stability on all mobile device testing
    • [ON TRACK] Deploy Autophone (Noah's Ark) mochitest and robocop test abilities.
  • Create and deploy performance automation for desktop, fennec, and B2G product benchmarks
    • [ON TRACK] Create Eideticker tests for B2G on panda boards and run them once a day
    • [ON TRACK] Monitor Datazilla webservices and ensure our accuracy in detecting performance regressions on a per-push basis
    • [ON TRACK] Deploy Datazilla per-push UI to make it easier for developers to dig into the causes of performance regressions
  • Update our bugzilla instance to 4.2 providing many benefits from UI to helping people keep track of bugs/projects from within Bugzilla.
    • [ON TRACK] Deploy 4.2 version of Bugzilla
    • [ON TRACK] Ensure that bug migration tools are in place for B2G so that we can share bugs with outside services and partners.
    • [ON TRACK] Complete the 'Sandstone' CSS update
    • [ON TRACK] Rework browserID <--> BMO integration
    • [ON TRACK] Deliver user and product dashboards
    • [ON TRACK] Deploy tracking flags extension database optimization
  • Make the process of sheriffing easier by updating TBPL and helping make it easier to handle the intermittent test problem
    • [ON TRACK] Aid with B2G changes so that B2G oranges tracked in Orange factor, B2g tests show up in TBPL
    • [ON TRACK] Make it easier to sheriff and diagnose issues through better notification and automatic suggestions of issues for intermittent failures
    • [DONE] Split mochitest browser-chrome from mochitest-other to improve parallelism and overall turnaround time.
    • [ON TRACK] Make orange factor update more than once a day to help make it easier to see changes over time
    • [ON TRACK] Assist with getting Datazilla's UI integrated with TBPL, phase 1
  • Expand and Modernize the QA Automation framework as well as make it easier and more clear how to write automated tests for QA.
    • [ON TRACK] Re-organize and improve the automated testing documentation at MDN
    • [ON TRACK] Develop first round of unit tests for WebRTC
    • [ON TRACK] Ensure all B2G device WebAPIs testable through emulators have comprehensive low level test coverage
  • Projects Defered due to pivoting hard toward B2G
    • [DEFER] Port Eideticker to win32 and run tests there once a day to measure win32 snappiness for desktop
    • [DEFER] Create pageload benchmark with networking shaper tests so that we can adequately track user-perceived performance across all devices and network types
    • [DEFER] Complete REST API integration with Bugzilla
    • [DEFER] Update selenium grid to use ESX on-demand to improve web QA capacity - start date pushed back due to B2G
    • [DEFER] Finish and deploy mozmill 2.0 changes to frameworks and existing QA tests - start date pushed back due to B2G
    • [DEFER] Autophone goal scaled back and started later
    • [DEFER] Talos work scaled back as well.
Retrieved from "https://wiki.allizom.org/index.php?title=Auto-tools/Goals/2012Q4&oldid=483679"