Confirmed users
3,816
edits
QA/TDAI/Goals/2009-Q2: Difference between revisions
→Test Development
No edit summary |
|||
| (6 intermediate revisions by the same user not shown) | |||
| Line 4: | Line 4: | ||
* Increase visibility and expand automation by completing the build integration work for JS Ref Test, Mozmill, and Maemo automation harnesses. | * Increase visibility and expand automation by completing the build integration work for JS Ref Test, Mozmill, and Maemo automation harnesses. | ||
** ''JS Ref Test is at risk, mozmill is ok, maemo is ok.'' | ** ''JS Ref Test is at risk, mozmill is ok, maemo is ok.'' | ||
* {{ | * {{done|}}Support the 1.9.2 feature and performance work with functional test writing and aid in detailing/specifying new performance metrics (work with Fx Front End and Platform). | ||
** {{ | ** {{done|}} ''We completed work with networking performance, it is unclear if help is needed for front end, and mobile perf work is ongoing.'' | ||
* Expand our test coverage by recruiting developers and community to write targeted tests in critical areas and by devoting one day every other week to publicly work on test development for all available harnesses. | * Expand our test coverage by recruiting developers and community to write targeted tests in critical areas and by devoting one day every other week to publicly work on test development for all available harnesses. | ||
** {{ | ** {{done|}} ''Have not done as good a job at this as we would have liked early in the quarter due to 3.5 slippage, but will make up for it toward the end.'' | ||
* Maintain and improve test tools to better enable people to do both manual and automated testing. | * Maintain and improve test tools to better enable people to do both manual and automated testing. | ||
** QAC, Mozmill, and the Results server are the tools we aim to update this quarter. | ** QAC, Mozmill, and the Results server are the tools we aim to update this quarter. | ||
| Line 18: | Line 18: | ||
''' Tactics:''' | ''' Tactics:''' | ||
* {{ | * {{done|}}Institute a measuring system for tinderbox realibility | ||
* {{ | * {{done|}}Debug the known, observed, random failures, create a plan for handling random failures going forward. | ||
* {{risk|}}Land JS Ref test framework {{bug|469718}} | * {{risk|}}Land JS Ref test framework {{bug|469718}} - ''at risk due to 3.5 slip, getting build resources is hard and this feature is coming in late'' | ||
* {{ok|}}Enable Build/Test infrastructure to run Mozmill tests (unit tests at least) | * {{ok|}}Enable Build/Test infrastructure to run Mozmill tests (unit tests at least) | ||
** {{ | ** {{miss|}}Have Mozmill tests running automatically, reporting to results server and/or staging tboxes (as appropriate) | ||
* {{ | * {{done|}}Integrate Maemkit with Tinderbox, hand off to build | ||
'''Other Harness Level Projects''' | '''Other Harness Level Projects''' | ||
* {{ | * {{done|}}Research and develop tools to run unittests on Windows Mobile. This includes remote launching and refactoring existing tools | ||
** Prototype in Q2 | ** Prototype in Q2 | ||
** formally finished in Q3 | ** formally finished in Q3 | ||
| Line 42: | Line 42: | ||
Goal 1: | Goal 1: | ||
* {{done|}}Be involved in planning discussions for our areas of focus | * {{done|}}Be involved in planning discussions for our areas of focus | ||
* {{ | * {{ok|}}Draft test priority documents detailing the new features we will concentrate on for functional testing, get at developer review of those | ||
* {{ | * {{miss|}}Write those functional tests | ||
** - ''3.5 slip has prevented us from concentrating as much here as we had anticipated'' | |||
* {{ | * {{done|}}Aid Mobile team as needed with their performance testing (seems like Taras is doing much of the heavy lifting?) | ||
* {{drop|}}Gather requirements and create mechanism for Firefox front end team's user performance metrics from "dirty" profiles - ''not sure how much this is still needed, and marking dropped as this item has been pushed into q3 by 3.5'' | |||
* {{done|}}Gather requirements and gather platform performance metrics surrounding network performance for 3.5 | * {{done|}}Gather requirements and gather platform performance metrics surrounding network performance for 3.5 | ||
| Line 54: | Line 55: | ||
*** {{done|}}Prioritize recently fixed security bugs for inclusion on the hot-spot list | *** {{done|}}Prioritize recently fixed security bugs for inclusion on the hot-spot list | ||
** {{done|}}Create the 'hot-spot' list for each area - JS, Content, Layout/Gfx, Mobile/Ffx Frontend | ** {{done|}}Create the 'hot-spot' list for each area - JS, Content, Layout/Gfx, Mobile/Ffx Frontend | ||
** {{ | ** {{drop|}}Bring the list to developers to get them to help create tests - ''With the unrelenting focus on code blockers for 3.5 we did not feel it appropriate to bug developers with this'' | ||
** Not having done this before, use a time based approach. Next time we will look at number of test cases written this quarter and see how to change this goal going forward | ** Not having done this before, use a time based approach. Next time we will look at number of test cases written this quarter and see how to change this goal going forward | ||
= Community Leadership = | = Community Leadership = | ||