QA/Automation/Meetings/110810

previous meeting | Meetings | next meeting »

Dial in

 # 650-903-0800 or 650-215-1282 x92 Conf# 315 (US/INTL)
 # 1-800-707-2533 (pin 369) Conf# 315 (US)
 # irc.mozilla.org #mozmill for backchannel

Attendees

• Henrik Skupin
• David Burns
• Dave Hunt
• Geo Mealer
• Owen Coutts
• David Clarke
• Marlena Compton
• Al Billings

Last Weeks Action Items

• Geo: Q2 achievements for shared API refactor
  • Posted here
• Need to add leads and seconds to list in agenda notes
  • Definition of project leads (1st and 2nd) [Henrik]

   Automated Testing for WebApps: Henrik / DavidC
   Team Presence (Documentation): DavidB / ?
   Infrastructure Investigation: ? / ?
   Selenium Grid 2: Dave / DavidB
   Endurance Tests: Dave / Anthony 

• need to setup a project meeting for open web apps
• everyone to review goals
• QA infrastructure - make this it's own sub-tree in the wiki pages
• Mozmill Broken tests - Geo to take point for gathering data and plan for delegation
  • Expected more tactical discussion during workweek on this; will schedule breakout meeting w/ Henrik & stakeholders

Roundtable

• Standup Day/Time (see the calendar)
• Next meeting:
  • Chair: Dave Hunt
  • Notes: Geo
• Update/release testing analysis and improvement (Geo)

Goals Overview 11/Q3

Please mark topics which need to be discussed in the meeting as bold

Issues / Blockers

• TBD.

Automated Testing for WebApps

• Initial Etherpad has been created. We are still unclear about strategies, milestones, and testing efforts.
• Created a Etherpad to keep track off all the conversations happening.
• Goal to setup a meeting with ragavan / jennifer arguello to determine roadmap / rough delivery plan.
• Experiments in test frameworks.
• Waiting for feedback from OWA developers

Team Presence (Documentation)

• Table of members has been created (Proposal for QA whois page)
• Next on the list are the project pages (Henrik)
  • Do we want to have quarterly project pages which map to our team goals?
    • Those will exist in parallel to the real project pages and only visualize what happens in the current quarter
    • That way we can give a bit more information as only adding the goal summary
    • It will give us a clearer picture for projects which span multiple quarters
• How can we share the other work to do across the team?
  • Creation of the wiki structure
  • Documents like responsibilities...

Infrastructure

• TBD.

Selenium Grid 2

• No updates.

Endurance Tests

• QA Desktop team is now tracking Endurance metrics
  • Spreadsheet
  • Done daily as part of regular triage -- escalate Firefox issues to release drivers, Endurance failures to Dave Hunt
  • Will do this manually until something automated can be rigged up

Project Updates

Please mark topics which need to be discussed in the meeting as bold

Infrastructure

• Shadow test server
• Cloud Automation

Case Conductor (TCM)

• TBD.

Mozmill

• New Tests
  • 9 Add-ons Manager tests in development
  • Softvision team now 4 automaters, up from 2 (will introduce once on-boarded)
• Broken Tests
  • NEW (12) | ASSIGNED (7) | FIXED (3)
• Shared Modules
• Redesign Shared Modules
  • Large merge landing this week
  • Driver (general framework) module sold to A-Team
  • On the lookout for a good UMLish class design collaboration tool -- know one?
• Automation Scripts
  • New daily test-runs
• Dashboard
  • New results view for remote test-run
• Mozmill Crowd
• On-demand Test Execution
  • New system that also allows for functional testing rolled out on qa-set
  • Works well, but have seen problems with shared filesystem. Not a factor of these scripts, but affects them since Python libraries not correctly loading.
  • Phase II is a one-click web front end. This is moving onto the hotseat (more in Round Table).

Selenium

• Grid
• Mobile

Other

• E.g. JMeter

Personal Status

For the personal status please check the weekly status updates:

• 110801 - 110805
• 110807 - 110812

Meeting Notes

• Last meeting's action items:
  • Review personal goals and how they align to the four top level goals
    • Endurance tests goal well on track
    • Web apps goal needs to be discussed
  • QA infrastructure wiki sub-tree with Al but he's been busy - target end of this week
  • Mozmill broken tests
    • Geo needs to schedule breakout to progress this

• Roundtable:
  • Standup schedule
    • Monday/Wednesday middle of the day Pacific time
    • Communication preferences: Vidyo, voice, IRC
  • Next meeting
    • Chair: Dave Hunt
    • Notes: Geo Mealer
  • Update/release testing analysis and improvement
    • Matt E sent email during the work week
    • Needs to be in the backlog as a high priority
    • Phase 1: complete
    • Phase 2: web front end and simplify result aggregation
      • Document what we have now (Geo, Al, Henrik, Anthony) and reviewed by all
      • Implement launcher page
      • Analyse process and discuss potential optimisations
    • This is a team project

• Goals Overview:
  • Automated Testing for WebApps
    • Henrik has created an Etherpad for web apps
    • Development team have a set of Q3 goals but not enough detail to make any decisions
    • Meeting set up for 08/11/2011 to go over schedule
    • Need an idea for when code is going to land across the different features
    • Get a skeleton testing framework in place early
    • Mobile/desktop will be targeted separately and will land independently
  • Team Presence
    • Henrik made a proposal during the work week for quarterly project pages that map to team goals
    • Suggest making a sample page to get an idea for maintenance and relevance

• Other:
  • Geo and Henrik looking for collaborative tools for class design
  • Why are our Pivotal Tracker trackers private?
    • All are public except our Automation Services project

See also: http://etherpad.mozilla.com:9000/TVNXcXuaM2

Action items

• [DONE] Send out an e-mail about assinging leads/seconds for projects. (Henrik)
• Schedule mid-quarter review (full team or individual reviews) for Automation Services goals. (David B)
• QA Infrastructure to be it's own sub-tree. (Al)
• Schedule breakout meeting for Mozmill broken tests. (Geo)
• Schedule standup for Monday/Wednesday during overlap availabilty times. (David B)
• Create a checklist for setting up standups. (David)
• Add meeting notes to a separate section of the wiki. (Dave)
• Add action items to a separate section of the wiki. (Dave)
• Document (and distribute) what automated update testing we have now by next meeting. (Geo, Al)
• Schedule meeting to define our approach to open web apps from a QA perspective. (DavidC)
• Create a one-stop references page for open web apps. (DavidC/Henrik)
• Send email with sample for quartlerly project pages, demonstrating level of maintenance required and relevance to the community. (Henrik)
• Start time-boxed e-mail thread to gather feedback on the suggestions for team wiki structure. (Henrik)
• [DONE] Makes sure all Pivotal Tracker projects are public. (Henrik)
• List Pivotal Tracker projects from wiki (project pages, or an index?). (Unassigned)
• Schedule meetings based on discussion items from work week. (David B)
• Create backlog from discussion items from work week. (David B)