QA/TDAI/2009 Q4 Sprints

From MozillaWiki
Jump to navigation Jump to search

We have three sprints in Q4 as you can easily see from https://wiki.mozilla.org/QA/TDAI/Goals/2009-Q4.

Sprint 1 Review: October 23

  • Mobile Test Framework
    • Quick Demo
    • Where we Are
      • Basic DeviceAgent with 90% functionality implemented
      • Working scripts for mochitest, xpcshell, and talos to utilize DeviceAgent
    • Next Steps
      • Heartbeat and DeviceInfo from Device->TestAgent
      • Integrate harness scripts into TestAgent, including download/install, verify, mochitest, xpcshell
      • Upload logs, status, and updates to TestServer
  • Valgrind Test Automation
    • (no demo -- too slow!)
    • Where we are
      • We have support for running valgrind with jsreftests, crashtests, reftests, xpcshell tests, and mochitests. All relevant patches have landed: bug 521130, bug 521457.
    • Next Steps
      • Collect valgrind exclusion list, for excluding certain uninteresting errors.
      • Investigate CPU/RAM requirements for VM's.
      • Investigate use of valgrind 3.2.1 (ships with CentOS 5) vs 3.5.
      • Investigate log parsing: XML output, error summaries.
  • Automatic Crash Reporting
    • Demo?
    • Where we are
    • Next Steps
      • How close are we to turning it on?
  • Electrolysis Testing
    • (no demo -- looks just like regular mochitest!)
    • Where we are
      • OOP Plugin Tests
        • Have a mochitest-ipcplugins make target for running OOP plugin tests: bug 519574. This is being tested for inclusion onto e10s tinderbox: bug 523712.
        • Nearly done with npruntime tests, still need tests for NPN_Construct and NPN_Evaluate - next week?
    • Next Steps
      • OOP Plugins Tests
        • Finish tests for NPN_Construct and NPN_Evaluate.
        • Dev moves OOP plugins onto moz-central, preffed off (bug 523094) after basic test framework changes are implemented (see below).
      • OOP Framework Changes
        • Mochitest and xpcshell test runners need to be made aware of multiple processes - in progress: bug 523208, bug 523211.
        • Need to figure out how to deal with hangs in child processes, maybe bug 501034 helps.
        • Crash detection probably works OK for multiple processes in the context of the python test runners, but need to test.
        • Need to figure out how automatic crash reporting is impacted by multiple processes (longer-term task).
        • Leak reporting in the context of the python test runners needs to be updated: bug 519570
      • OOP Talos - owner unclear
        • New plugin tests need to be implemented for plugin-related metrics: bug 519568.
        • Talos needs to be updated to expect/handle multiple processes.
      • OOP Content Tests
        • Need to attempt running all unittests with OOP content enabled, and identify problems - may be blocked pending additional necko and docshell work.
        • Fix mochitests which fail under OOP content because they use elevated privileges in the "wrong" place.
  • XBL 2
    • Demo
    • Where we are
    • Next Steps (beyond Q4)
  • Jetpack Test Harness
    • Demo
    • Where we are
    • Next Steps
  • Log Compare
    • (no demo)
    • Where we are
      • Fennec?
      • Firefox?
    • Next Steps
      • Fennec
      • Firefox
  • Adobe Test Harness Automation
    • (no demo)
    • Where we are
      • Waiting on Adobe for links/access
    • Next Steps
      • Jgriffin will own getting it running and automated if we ever hear back from Adobe
  • HTML 5 W3C Tests
    • (no demo)
    • Where We Are
      • Someone from Microsoft has been appointed to the position of Task Force Coordinator
      • Waiting on W3C to start doing stuff
    • Next Steps

Things Needing Attention

  • Gfx test harness
    • URGENT:: Need: Mechanism to measure "choppiness" of video for full screen tests
    • Need: Mechanism to test hardware accelerated 3d via extension
    • Need: Research into types of tests to write
    • Need: Research into WebGL implementation
  • E10S Talos integration
    • Can we get help from RelEng?
    • Does Jmaher have to own?
  • JS Ref Test
    • Browser Tests: Can we get this on Tinderbox quicker? bug 519213 - I believe after bug 487689 Split Mochitest lands we will be able to get more traction.
    • Shell Tests: Can we get these completed quicker? bug 509629 Waiting on dmandelin to finish working on crashkill top crashers.

Sprint 2 Review: November 20

Sprint 3 Review: December 18

Retrieved from "https://wiki.allizom.org/index.php?title=QA/TDAI/2009_Q4_Sprints&oldid=177817"