QA/Automation/Meetings/110713

previous meeting | Meetings | next meeting »

Dial in

 # 650-903-0800 or 650-215-1282 x92 Conf# 315 (US/INTL)
 # 1-800-707-2533 (pin 369) Conf# 315 (US)
 # irc.mozilla.org #mozmill for backchannel

Attendees

• Cameron, Anthony, Aaron, Henrik, Dave, Stephen, Al, Geo, Matt, Marlena, DavidC, DavidB

Last week's action Items

• Geo: Deliver summary for API refactoring and test porting project by latest end of this week
  • [MISSED] Still owe this. Will send to Henrik for review first, then forward to team.
• Anthony: Deliver summary for Discover Pane project by latest end of this week
• [DONE] Henrik: Work with Softvision to ensure that DiscoPane tests will be checked in latest by Friday
• All: Reply to the discussion email about auto-skipping DiscoPane tests
• Matt: update QA overall goals
• [DONE] Henrik/David: Finalize Q3 goals and get team feedback
• [DONE] Henrik/David: Work out preliminary wiki pages for QA Automation Services team (incl. meeting page)
• Henrik/David: Come up with the first draft of the team structure and tasks

Roundtable

• Work Week
  • Have all flight details been set? [Henrik]
• Definition of project leads (1st and 2nd) [Henrik]
  • Automated Testing for WebApps: Henrik / DavidC
  • Team Presence (Documentation): DavidB / ?
  • Infrastructure Investigation: ? / ?
  • Cross-Framework Test Creation: ? / ?
  • Selenium Grid 2: ? / DavidB
  • Endurance Tests: Dave / Anthony

Goals Overview 11/Q3

Automated Testing for WebApps

• Initial meeting with WebApp developers happened yesterday. Discussed roadmap of webapps and possible testing scenarios.
• ShaneT will come up with some functional use cases by end of this week
• It will become a challenging project!

Team Presence (Documentation)

• Initial pages (brainstormed ideas) will be available for review soon at http://wiki.mozilla.org/QA/Automation_Services
• Next week we want to start with brainstorming ideas for useful content, so we can safe that time in our work week

Cross-Framework Test Creation

• Will be dropped from the Q3 goals
• First idea is to be active on that project during an automation testday to show collaborated efforts across frameworks.

Selenium Grid 2

• We have upgraded to Selenium 2.0.0 but discovered a bug with session handling. We can't replicate it locally, but are working with François Renaud (main contributor for Selenium Grid) and hope to have a fix soon.

Endurance Tests

• Micro-iterations have landed -- This allows tests to accumulate within an iteration. For example, the open new tabs tests can now open several tabs within an iteration instead of just one.
• Working on a single ad-hoc test based on Mem Buster to give an indication of Firefox performance. bug 669613
• First community created endurance test is in review and should land tonight! bug 670721

Project Updates

Please mark topics which need to be discussed in the meeting as bold

Infrastructure

• Shadow test server
  • Waiting for IP addresses from rackspace that IT requested.
• Cloud Automation
  • No updates right now.
• Others
  • Talking to Zandr in IT about rackable server buildout to replace Mac Pros for day to day automation support.
    • Will be sending proposal/question e-mail to QA Automation team for comments.
  • What's the place on the wiki for infrastructure projects documentation?

Case Conductor (TCM)

• Environment Management UI was demo'd. Currently being implemented (slated for this week)
• hoping to get batch test case entry UI working. Given... When... format at first
• devs go on vacation for 1 month starting this weekend. In the interim, QA will be working on,
  • User feedback, surveymonkey
  • Automation connector
  • Limus migration tool

Mozmill

• On-demand Test Execution
  • Patch with final set of agreed-upon changes and deployment on qa-* boxes this week.
  • Should be usable for releases next week.
• Redesign Shared Modules
  • Patch with final set of window handler changes in progress, should be in shortly after this meeting.
• Mozmill Crowd
  • Mozmill Crowd 0.1.3 has been released
• Dashboard
  • Working on user stories as outlined http://etherpad.mozilla.com:9000/mozmill-dashboard-elasticsearch
  • Working on implementing per user features (logging in, commenting, triage)
• Tests / Shared Modules
  • Discovery Pane - 3 tests checked in across all branches, 2 tests checked in on release and beta, 3 tests in review
  • Addons Manager Restart - 3 tests assigned
  • AppTabs - 1 test checked in across all branches, community contributed (Shriram)
  • Endurance - Softvision to contribute 4 flash video tests, Shriram porting AppTabs test
• Broken Tests
  • We have some test failures which needs investigation and a fix
• Automation Scripts / Infrastructure]

Selenium

• Grid
• Mobile
• Browser
  • Selenium works with OSX 10.7
  • https://bugzilla.mozilla.org/show_bug.cgi?id=650858 is a blocker for Fx6 support for Selenium 2 API
  • Prepping for Se 2.1 release tomorrow or Friday
  • https://bugzilla.mozilla.org/show_bug.cgi?id=670674 - Selenium WebDriver integration for Firefox

Other

• Cross-weave / Funkload automated nightly testing
  • Creating a CRON job that runs smoke tests against the latest builds of the sync server
  • Basic reporting to an HTTP site for members of the sync team
• E.g. JMeter

Personal Status

For the personal status please check the weekly status updates:

• 110103 - 110107
• 110110 - 110114

Meeting Notes

• Last weeks Action Items
  • geo and anthony last weeks items are follow up to missed q2 goals
    • AI: put text in Q2 goals
• Roundtable
  • Workweek
    • need to add travel info into the sync pad:
      • coordinate travel arrangements
      • cottage is booked through till the 30th
    • AI: Need to add leads and seconds to list in agenda notes (everyone)
  • cross framework test creation
    • this isn't a team goal, but a good personal one
    • AI: include combined frameworks test creation into a automation testday (?)
• open web apps discussion
  • need to define this as a major project focus group
  • AI: need to setup a project meeting for open web apps (matt)
    • Henrik, David C, Marlena & Geo preliminarily assigned as the webapps team
    • POC (Mozmill)
    • #openwebapps IRC channel
    • Webapps Engineering meeting occurs on Tues @ 10a
      • AI: David Clarke to attend next Webapps meeting (David C)
• QA Automation Services Goals
  • AI: remove the cross framework test development goal as a toplevel team goals (Henrik S)
  • AI: everyone to review goals (everyone)
• Wiki Pages
  • will there be redirects for old pages?
  • QA infrastructure
    • AI: make this it's own sub-tree in the wiki pages (Al)
• Mozmill broken tests
  • do we have a list of failures?
  • broken tests can be found in dashboard
  • what about skipped tests?
  • whiteboard entry in bugs assigned for broken tests
  • henrik to add query to meeting notes
  • AI: Geo to take point for gathering data and plan for delegation (Geo)

Action items

• see bold lines above