QA/Test Automation/2010-09-22: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(→‎Personal: Adrian)
 
(15 intermediate revisions by 5 users not shown)
Line 7: Line 7:


=Attendees=
=Attendees=
* Adrian, Henrik, Anthony, Geo, Matt, Al, Aaron


= Last weeks action Items =
= Last weeks action Items =
* Geo: Send out list of broken tests w/ Fx4 and MM1.5
* Geo: {{miss|}} Send out list of broken tests w/ Fx4 and MM1.5
* Matt: Check what limits test harnesses on buildbot to use other local webservers (connect with John O'Duin)
* Matt: {{done|}} Check what limits test harnesses on buildbot to use other local webservers (connect with John O'Duin)
* Matt: Action items from release tasks meeting? Need to notify release leads that they're responsible for coordinating resources.
* Matt: {{miss|}} Action items from release tasks meeting? Need to notify release leads that they're responsible for coordinating resources.
* Geo: For Auto-tools meeting next wk, how to incorporate train model going forward
* Geo: {{done|}} For Auto-tools meeting next wk, how to incorporate train model going forward
* Henrik & Matt: How to streamline patch checkin process
* Henrik & Matt: {{miss|}} How to streamline patch checkin process
* Geo: Generate wiki page to sum up tab candy objective
* Geo: {{miss|}} Generate wiki page to sum up tab candy objective


=Goals Overview=
=Goals Overview=
Line 26: Line 27:
** Adrian: DOM walker and set of l10n tests  
** Adrian: DOM walker and set of l10n tests  
* Planning for Q4 (proposals)
* Planning for Q4 (proposals)
** [https://intranet.mozilla.org/2010Q4Goals#QA QA Q4 goals]
** Henrik: First public version of the Mozmill Crowd extension
** Henrik: First public version of the Mozmill Crowd extension
** Henrik: Dashboard for software updates and addons tests  
** Henrik: Dashboard for software updates and addons tests  
Line 42: Line 44:
** Creation of a PoC for Panorama tests during the QA workweek + Demo
** Creation of a PoC for Panorama tests during the QA workweek + Demo
* DOM Walker + L10n API (Adrian)
* DOM Walker + L10n API (Adrian)
** see my personal notes
* Redesign Shared Modules (Geo)
* Redesign Shared Modules (Geo)
** Got first "refactored" test working
** Got first "refactored" test working


;Local Test Data (Anthony)
;Local Test Data (Anthony)
* Likely to slip
* 4 r?, 11 checkin-needed, 10 need patches
* 4 r?, 11 checkin-needed, 10 need patches
* Won't have time to resolve all open issues and work on Panorama at the same time


;Broken Tests
;Broken Tests
Line 95: Line 96:
** {{done|}} Patch checkins to 0
** {{done|}} Patch checkins to 0
* Next two weeks
* Next two weeks
** Review new patches to 0
** Review new patches to 0 (3 remain)
** Commit new check-ins to 0
** Commit new check-ins to 0 (11 remain)
** Contribute 1 Panorama test
** Ensure 10 remaining local-data tests complete
** <strike>Contribute 1 Panorama test</strike>
** {{risk|}} QA results dashboard (if/when ready)
** {{risk|}} QA results dashboard (if/when ready)
** {{risk|}} QA crowd extension (if/when ready)
** {{risk|}} QA crowd extension (if/when ready)
** {{risk|}} Contribute 10 remaining local-data tests


;Al
;Al
Line 112: Line 113:
* Last two weeks:
* Last two weeks:
** [http://hg.mozilla.org/users/gmealer_mozilla.com/mozmill-refactor/file/682657b71106/firefox/testAwesomeBar/testAccessLocationBar.js refactored testAccessLocationBar.js], functional
** [http://hg.mozilla.org/users/gmealer_mozilla.com/mozmill-refactor/file/682657b71106/firefox/testAwesomeBar/testAccessLocationBar.js refactored testAccessLocationBar.js], functional
*** Was functional as of last night, minorly broken again right now on a module chaining issue.  Will have code pushed to personal hg as soon as that's fixed.
** {{prev|}} collecting team feedback on prototype
** {{prev|}} collecting team feedback on prototype
** {{prev|}} Fixes for {{bug|573582}}, {{bug|573584}}, {{bug|575250}}
** {{prev|}} Fixes for {{bug|573582}}, {{bug|573584}}, {{bug|575250}}
*** {{bug|575250}} fixed
** Presentation w/ Aaron
** Presentation w/ Aaron
*** Rough draft (info dump, really) created at [https://docs.google.com/present/edit?id=0AXEJ45MnstYjZGhkcGRkNHFfN2hyZzczaHN2&hl=en&authkey=CKzF3agM Google Docs]
** QA work week
** QA work week
*** Finished first rev of performance checkpointing module
*** Demoed for Tab Candy team
*** Much hilarity had with all
* Next two weeks:
* Next two weeks:
** Two more tests for refactor project, one from AwesomeBar, one from elsewhere.
** Bug fixes for broken 4.0 tests
** Any other necessary work to fulfill team goals


;Aaron
;Aaron
Line 129: Line 139:
;Adrian
;Adrian
* Last two weeks:
* Last two weeks:
** {{prev|}} finish Documentation + UML Diagrams for CommonA11yL10nAPI and L10nAPI
** {{done|}} (mostly) finish Documentation + UML Diagrams for CommonA11yL10nAPI (renamed to: DOMUtilitiesAPI) and L10nAPI
** {{prev|}} implementation of the CommonA11yL10nAPI and L10nAPI
** {{done|}} (mostly) implementation of the CommonA11yL10nAPI (renamed to: DOMUtilitiesAPI) and L10nAPI: WIP patch attached to {{bug|562084}} - including two tests for the preferences window
* Next two weeks:
* Next two weeks:
** {{done|}} (mostly) finish Documentation + UML Diagrams for CommonA11yL10nAPI (renamed to: DOMUtilitiesAPI) and L10nAPI
** implement the L10n tests for the update window
** {{done|}} (mostly) implementation of the CommonA11yL10nAPI (renamed to: DOMUtilitiesAPI) and L10nAPI: WIP patch attached to Bug 562084 - including two tests for the preferences window
** bring the 2 shared-modules and 4 tests through the reviews


=Contributor Status=
=Contributor Status=
Line 148: Line 158:
** Requested 32GB for both machines ({{bug|598305}}) - Matt has to approve
** Requested 32GB for both machines ({{bug|598305}}) - Matt has to approve


=Action Items=
=Meeting Notes=
* {{prev|}} Geo: Send out list of broken tests w/ Fx4 and MM1.5
* Last action items
** Limits of the test harness on buildbot to use other local webservers
*** Pretty strict handling for accessing other servers
*** It would be possible to have tests running on a Talos server; Therefore a test environment has to be setup
* Goals Q3
** We drop the Mozmill crowd extension from our list
** Dashboard will be ready for general test results by end of this week
** Anthony and Aaron will finish the first part of the local test data project 
** Henrik updates the AddonsAPI and the existing tests until end of next week (Aaron will help out once the local data project is done)
** Geo will have a solid plan for the refactoring work by end of this month
** Adrian will work on the DOMWalker to have it ready by end of this month
* Goals Q4 (Finalized by next week)
** {{prev| Develop and deploy a Mozmill crowd-sourcing extension that will allow community users to execute and report results of Firefox 4 mozmill automated tests across the available platforms and configurations used by the  Mozilla  testing community. }}
** {{new|Integrate local data-based Firefox 4 mozmill tests under Buildbot execution control and reporting}}
** {{new|Collaborate with development team to expand Mozmill Panorama support to include an automated test suite for functional and performance testing of the Firefox 4 Panorama feature.}}
** {{new|}} Implement Mozmill Dashboard views for software update and addon tests
** {{new|}} Refactoring the shared module system based on the specification from Q3
* Others
** Geo: More often conversations about goals and blockers except our 1-1's and the bi-monthly meeting
*** We will setup our own IRC #mozmill channel and have an internal mailing list (qa-auto)
** On demand test execution improvements
*** Geo: Shouldn't be a team goal regarding time constraints
*** Henrik: We can have at least the download script integration
 
==Action Items==
* Geo: {{prev|}} Send out list of broken tests w/ Fx4 and MM1.5
* Geo: schedule in-IRC meeting to do patch-handling proposal
* Geo: {{prev|}} Generate wiki page to sum up tab candy objective
* Henrik: Talk with John O'Duin about specifics of the test environment
* Henrik: {{done|}} Setup IRC channel #mozmill + Mailing list (qa-auto) [{{bug|598883}}]
* Henrik: {{done|}} Send out goals proposals for final rework
* Matt: {{prev|}} Action items from release tasks meeting? Need to notify release leads that they're responsible for coordinating resources.

Latest revision as of 16:22, 27 September 2010

previous meeting | Meetings | next meeting »

Dial in

 # 650-903-0800 or 650-215-1282 x92 Conf# 315 (US/INTL)
 # 1-800-707-2533 (pin 369) Conf# 315 (US)
 # irc.mozilla.org #qa for backchannel

Attendees

  • Adrian, Henrik, Anthony, Geo, Matt, Al, Aaron

Last weeks action Items

  • Geo: [MISSED] Send out list of broken tests w/ Fx4 and MM1.5
  • Matt: [DONE] Check what limits test harnesses on buildbot to use other local webservers (connect with John O'Duin)
  • Matt: [MISSED] Action items from release tasks meeting? Need to notify release leads that they're responsible for coordinating resources.
  • Geo: [DONE] For Auto-tools meeting next wk, how to incorporate train model going forward
  • Henrik & Matt: [MISSED] How to streamline patch checkin process
  • Geo: [MISSED] Generate wiki page to sum up tab candy objective

Goals Overview

  • Current status for Q3 goals
    • All: Set of tests for a main feature of Firefox 4
      • Can we make it an accomplished goal?
      • What takes lesser resources: Addons Manager vs. Panorama
    • Henrik: Dropping Mozmill Crowd extension?
    • Henrik: Web dashboard will be usable for the general test-run
    • Geo: Model for re-designing the shared modules
    • Adrian: DOM walker and set of l10n tests
  • Planning for Q4 (proposals)
    • QA Q4 goals
    • Henrik: First public version of the Mozmill Crowd extension
    • Henrik: Dashboard for software updates and addons tests
    • Henrik: Continue preparation of tests to get run on buildbot (incl. local data and stablization)
    • Henrik: Continue creation of tests for Panorama (performance?)
    • Henrik: Refactoring the shared module system
    • Henrik: Enhancements to the on demand testing infrastructure

Projects

Mozmill (Henrik)
  • Buildbot integration with 10 tests will happen today or tomorrow
  • Common JS module for a more streamlined module inclusion will be available with Mozmill 1.5.1 (smooth transition for Mozmill 2.0)
Test creation / shared modules
  • General (Henrik)
    • Creation of a PoC for Panorama tests during the QA workweek + Demo
  • DOM Walker + L10n API (Adrian)
    • see my personal notes
  • Redesign Shared Modules (Geo)
    • Got first "refactored" test working
Local Test Data (Anthony)
  • 4 r?, 11 checkin-needed, 10 need patches
Broken Tests
  • Firefox 4.0 (Geo)
  • Firefox 3.5/3.6 (Anthony)
    • No update
Update Tests (Henrik)
  • Fixed problem with missing fallback string in wiki results
  • Updated the SoftwareUpdateAPI for recent changes on trunk
  • bug 597664 gives us the chance to track slow mirrors
MozMill Crowd extension (Henrik)
  • No update
Add-ons Testing (Henrik)
  • Working with M.D.-Arche on Fidesfit client tests (bug 570493
Automation (Henrik)
  • No update
Dashboard (Henrik)
  • Staging server
  • Problems with filtering can be solved by using Ajax for view requests

Risky Goals

  • Mozmill Crowd extension

Personal

Henrik
  • Last two weeks
    • [DONE] Attended CouchCamp
    • [DONE] QA workweek - Creation of a PoC for Panorama + work on presentation
    • [MISSED] Finish work on the dashboard (general testrun)
    • Fixes for TabbedBrowsingAPI and SoftwareUpdateAPI
  • Next two weeks
    • Finish work on the dashboard (general testrun)
    • Depending on the goals we want to achieve this quarter:
      • Continue work on the crowd extension
      • Update AddonsAPI and prepare list of Mozmill tests to implement
Anthony
  • Last two weeks
    • [DONE] Patch review queue to 0
    • [DONE] Patch checkins to 0
  • Next two weeks
    • Review new patches to 0 (3 remain)
    • Commit new check-ins to 0 (11 remain)
    • Ensure 10 remaining local-data tests complete
    • Contribute 1 Panorama test
    • [AT RISK] QA results dashboard (if/when ready)
    • [AT RISK] QA crowd extension (if/when ready)
Al
  • Last two weeks
    • Work on tab candy as possible during work week.
    • Finish carryover goals.
  • Next two weeks
    • Going to Bali.
Geo
  • Last two weeks:
    • refactored testAccessLocationBar.js, functional
      • Was functional as of last night, minorly broken again right now on a module chaining issue. Will have code pushed to personal hg as soon as that's fixed.
    • [CARRY OVER] collecting team feedback on prototype
    • [CARRY OVER] Fixes for bug 573582, bug 573584, bug 575250
    • Presentation w/ Aaron
    • QA work week
      • Finished first rev of performance checkpointing module
      • Demoed for Tab Candy team
      • Much hilarity had with all
  • Next two weeks:
    • Two more tests for refactor project, one from AwesomeBar, one from elsewhere.
    • Bug fixes for broken 4.0 tests
    • Any other necessary work to fulfill team goals
Aaron
  • Last two weeks:
    • [CARRY OVER] of conversion of 31 tests to use local-data (patches and fixes upon reviews)
    • Attend QA work week
      • Presentation work w/Henrik & Geo
      • Panorama Challenge w/Henrik & Geo
      • local-data project work
  • Next two weeks:
    • Either divert focus to creating BFT focused Panorama tests with Anthony or carry on with local-data
Adrian
  • Last two weeks:
    • [DONE] (mostly) finish Documentation + UML Diagrams for CommonA11yL10nAPI (renamed to: DOMUtilitiesAPI) and L10nAPI
    • [DONE] (mostly) implementation of the CommonA11yL10nAPI (renamed to: DOMUtilitiesAPI) and L10nAPI: WIP patch attached to bug 562084 - including two tests for the preferences window
  • Next two weeks:
    • implement the L10n tests for the update window
    • bring the 2 shared-modules and 4 tests through the reviews

Contributor Status

  • Name

Roundtable

General
Issues
  • Brasstacks
    • Issues with vhost/reversed proxy will remain for a while until VM's can be moved to Colo in Phoenix; AT team has to wait for Releng and prepare the move. Will be solved next quarter.
  • QA-Horus / QA-Set
    • Too less memory when running all VM's in parallel
    • Requested 32GB for both machines (bug 598305) - Matt has to approve

Meeting Notes

  • Last action items
    • Limits of the test harness on buildbot to use other local webservers
      • Pretty strict handling for accessing other servers
      • It would be possible to have tests running on a Talos server; Therefore a test environment has to be setup
  • Goals Q3
    • We drop the Mozmill crowd extension from our list
    • Dashboard will be ready for general test results by end of this week
    • Anthony and Aaron will finish the first part of the local test data project
    • Henrik updates the AddonsAPI and the existing tests until end of next week (Aaron will help out once the local data project is done)
    • Geo will have a solid plan for the refactoring work by end of this month
    • Adrian will work on the DOMWalker to have it ready by end of this month
  • Goals Q4 (Finalized by next week)
    • [CARRY OVER] Develop and deploy a Mozmill crowd-sourcing extension that will allow community users to execute and report results of Firefox 4 mozmill automated tests across the available platforms and configurations used by the Mozilla testing community.
    • [NEW] Integrate local data-based Firefox 4 mozmill tests under Buildbot execution control and reporting
    • [NEW] Collaborate with development team to expand Mozmill Panorama support to include an automated test suite for functional and performance testing of the Firefox 4 Panorama feature.
    • [NEW] Implement Mozmill Dashboard views for software update and addon tests
    • [NEW] Refactoring the shared module system based on the specification from Q3
  • Others
    • Geo: More often conversations about goals and blockers except our 1-1's and the bi-monthly meeting
      • We will setup our own IRC #mozmill channel and have an internal mailing list (qa-auto)
    • On demand test execution improvements
      • Geo: Shouldn't be a team goal regarding time constraints
      • Henrik: We can have at least the download script integration

Action Items

  • Geo: [CARRY OVER] Send out list of broken tests w/ Fx4 and MM1.5
  • Geo: schedule in-IRC meeting to do patch-handling proposal
  • Geo: [CARRY OVER] Generate wiki page to sum up tab candy objective
  • Henrik: Talk with John O'Duin about specifics of the test environment
  • Henrik: [DONE] Setup IRC channel #mozmill + Mailing list (qa-auto) [bug 598883]
  • Henrik: [DONE] Send out goals proposals for final rework
  • Matt: [CARRY OVER] Action items from release tasks meeting? Need to notify release leads that they're responsible for coordinating resources.
Retrieved from "https://wiki.allizom.org/index.php?title=QA/Test_Automation/2010-09-22&oldid=255970"