QA/Test Automation/2011-06-29

previous meeting | Meetings | next meeting »

Dial in

 # 650-903-0800 or 650-215-1282 x92 Conf# 315 (US/INTL)
 # 1-800-707-2533 (pin 369) Conf# 315 (US)
 # irc.mozilla.org #mozmill for backchannel

Last action items

• David: Work with Geo and Anthony to get some first ideas for changes of the review process; we will schedule a meeting once Henrik is back (should happen in June)
  • We should have a deeper discussion during our upcoming work week (Henrik)

Goals Overview

• Revisit our Automation Team goals for 11/Q2
• Work on Automation Services goals for 11/Q3 (proposals)

Risky Goals

• Complete the refactoring of the API
• Create the top 5 tests for the discovery pane
  • 4 tests now in SR, remainder need revisions

Project Status

Redesign Shared Modules (Geo/Henrik)

• Window object model changing from a mixin-based model to an inheritance based model. Is in debug now, code will be delivered today.
• Known state implemented for test start/end: fresh default bookmarks, no tab history, all tabs closed except about:blank. Default tabs can be left for test start if desired (probably only for the test that verifies the default tab set). Still need to add "close all open windows but the original," will be settled out once model change is in.
• Windowing is the last fundamental being worked on. In-content windows (addons manager, notably) still unaddressed, bumped to next quarter.
• Implicit waitFor added on element finding (and is there for window finding as well).
• Test areas and top tests have been triaged and prioritized and added to Pivotal.

On-demand release tests (Geo/Henrik)

• Henrik is working on the final version of the staging script, which will also cover candidate builds (and later daily builds)
• Geo is working on adding BFT kickoff to the daemons, as well as code cleanup per Henrik's code review.

Endurance Tests (Dave/Anthony)

• Flash videos landed on mozqa.com and litmus-data, will target test page and test case writing next quarter
• Waverley to assist next quarter by making a portion of their test development Endurance tests related to the focus area

Others

Mozmill Crowd

• We are about to release Mozmill Crowd 0.1.3 by latest tomorrow (bug 667902)
• Waiting for preliminary review on addons.mozilla.org
• Testday for Mozmill Crowd (and test development) this Friday, July 1st

Dashboard

Current dashboard installation (Henrik)

• no updates

New dashboard driven by ES (Owen)

• updates?

Sync Server Load Testing (Owen)

• updates?

On Demand Provisioning (Cloud Automation) (Al/Henrik)

• Vsphere/ESX box is fully operational
• Al is creating reference virtual machines on it from scratch (porting existing VMs is impossible...mostly)
  • First post-reference VM will be a squid proxy
• What specific needs do people have here for on demand machines?

Tests / Shared Modules

General (Henrik)

• Check-ins since the last meeting

Broken Tests

• A couple of broken tests across brances in the last days. It has been caused by Henrik's patch for the Mozmill change in handling 'documentLoaded'. As is turned out today it's a Mozmill bug which blocks the thread execution if the window passed to the controller is null. See 668202 bug 668202.

Softvision (Anthony)

• 4 Discovery Pane tests in SR, remainder need revisions
• Will focus on Add-ons Manager restart tests next quarter, tacking on Web Apps or a Firefox 8 feature pending feature readiness

TCM (Cameron/Rebecca B.)

• .4 progress hampered a bit by platform instability. We will not make all features by 7/15.
• Negotiating with uTest on process and staffing improvements on their side.
• Hashing out task assignments for Rebecca and Matt B.
• Work on Environment management continuing.

Shadow web server (Al/Henrik)

• updates?

Automation Scripts / Infrastructure (Henrik)

• Download script has been enhanced to also download daily builds of Nightly or Aurora
• We can now even download Nightly builds from project branches - anyone interested in?

Personal Status

For the personal status please check the weekly status updates:

• 110620 - 110624
• 110627 - 110701

Roundtable

General

• Changes to our bi-weekly meeting regarding the org changes (Henrik)
  • We should setup preliminary pages which describe our new team next week
  • We should move the meeting pages below the new team page and call it "QA Automation Services Meeting"
• Review process (Henrik)
  • A more detailed discussion should happen during our work week in Cambridge
  • We have to find an interim solution to get not stuck in review requests
    • If you are not able to react to a review request from Softvision or contributors within one day, you should pass the review to another person
    • If there are issues you are running into, spread this information across the team. It takes lesser time to solve it together
• Discovery Pane tests
  • Do we want to allow tests to auto-skip themselves? If not we should mark those tests as skipped on default and aurora as long as the dependencies with WebDev are not solved.