QA/Automation/Meetings/141222

previous meeting | Meetings | next meeting »

Dial in

 # When:     Every Monday at 8:45am PDT/PST
 # Vidyo:    https://v.mozilla.com/flex.html?roomdirect.html&key=PGtLpx3XQGJz
 # Phone:    650-903-0800 or 650-215-1282 x92 Conf# 9654 (US/INTL)
 #           1-800-707-2533 (pin 369) Conf# 9654 (US)
 # IRC:      irc://irc.mozilla.org:6697/#automation
 # IRC logs: http://irclog.gr/#browse/irc.mozilla.org/automation

General

• Attendees:
• Chair: Henrik
• Notes: Andreea

Last Weeks Action Items

• None! Hurray!

Highlights

• [henrik] Firefox Automation report – week 41/42 2014: http://www.hskupin.info/2014/12/17/firefox-automation-report-week-41-42-2014/
• [henrik] Firefox Automation report – week 43/44 2014: http://www.hskupin.info/2014/12/18/firefox-automation-report-week-43-44-2014/
• [henrik] Firefox Automation report – week 45/46 2014: http://www.hskupin.info/2014/12/19/firefox-automation-report-week-45-46-2014/
• [henrik] Started work towards a Mozmill 2.1 release (upgraded mozbase packages, fixed manifest handling for parent, fixed disconnect failure)
  • there were a couple of fixes, detailed below in the Mozmill section
• [henrik] Started to mass-close Mozmill bugs due to the transition to Marionette (60 bugs closed - about 60 remain)
• [henrik] mozmill-ci on staging got updates: https://github.com/mozilla/mozmill-ci/compare/d5762e21dde4...edce13666d18 (Ubuntu 14.10)

Blockers

• List of blockers (http://bit.ly/Tph0NY):
  • [07/10] - Bug 735184 - RFE: Create Pulse notifications for update channel activities (goal for Q1/2015)
  • [09/03] - Bug 997230 - Setup Mac Mini for DeployStudio needed by PuppetAgain (needs action RelOps)
  • [10/29] - Bug 917204 - Add open squid proxy to qa.scl3.mozilla.com for testing purposes (needs testing)
    • needs testing for ssl, for blue and green larry
  • [10/29] - Bug 1066493 - TPS failure: Error: signIn() failed with: "error.sync.reason.serverMaintenance" (would need fix from DB vendor)
    • the workaround we tried is not really working so we're waiting for a db fix
  • [11/05] - Bug 1083166 - [SCL3] Pulse and maybe other connections are timing out in irregular intervals (maybe also causing test failures when loading web pages)
  • [11/26] - Bug 1105356 - [pulsetranslator] Missing normalized build notifications for Firefox 34.0.5/34.1 release across platforms and locales
  • [12/09] - Bug 1108313 - Slow network for Firefox 35.0b1 update tests (beta channel) in qa.scl3.mozilla.com
  • [12/03] - Bug 1107405 - Doorhanger notifications don't show up only before Firefox closes
    • Cosmin needs to continue here

Contributor Spotlight

• TBD

Roundtable

• Goals Q1 2015: https://etherpad.mozilla.org/QA-Goals-2015Q1
• Convert P1 Update and l10N tests from MozMill to marionette test framework
  • those will be first, then functional and remote and we'll have to see if we want addons
  • Henrik is working on modules and the other stuff remaining, a lot of work
• Get Marionette green light tests running in the Mozmill CI systems
  • we'll do this branch by branch, will ride the trains
  • for the other branches we'll continue with mozmill, it might be a bit tricky but it should work as each testrun has its own config so we can choose if to use mozmill or marionette
• Finalize release process management (update tests, specifically)
  • [henrik] proposal is up here: https://github.com/mozilla/mozmill-ci/issues/535
  • update tests should be fully automated, should be no QA person to trigger the ondemand testruns, all should be done via pulse

Goals

• https://wiki.mozilla.org/QA/Automation#Goals

Framework and Tools (Henrik)

Make Marionette ready for Mozmill tests

• Working on the integration for new the chrome window feature of Marionette (see Github)
• Continuing with Observer and Event classes for handling opening/closing of chrome windows, and once done for tabbed browsing

Locale Rotation for Aurora builds in Mozmill-CI

• No updates

Turn on PuppetAgain for Mozmill CI and TPS CI

• No updates

Improvements to Testday bot [stretched]

• First changes from Barbara will land hopefully today - should make the bot working
• More follow-up patches to come the next days

Test Automation (Softvision)

Creation of automated tests for important regressions

Fixed:

• bug 1088561 - Add automated test for plugins in different states
• bug 1088638 - Add automated test to install, disable, enable and remove services

In progress:

• bug 1098351 - Add new automated test for "Override mixed script content blocking"
• Teodor has 2-3 tests for about:newtab tiles in progress too

Project Updates

Mozmill (Henrik)

• Started to mass-close Mozmill bugs due to the transition to Marionette (60 bugs closed - about 60remain)
• Mozmill 2.1 work has been started
  • Fix for "client disconnected" failure has been landed (bug 794020)
  • we kill the process now, we can see that now in the exit code (-9)
  • Manifestparser fixes for parent manifests (bug 1056045), and regression for defaults (bug 1110837) landed
  • we now have fully support for parent manifest, the issue we saw last week got fixed (the one that skipped all tests if the first test in the manifest was disabled)
  • mozbase packages have been updated (bug 1111682) after additional fixes for mozrunner regressions
• New Ubuntu 14.10 nodes are running in Mozmill CI staging - if all is fine we will enable those on production next week
• Henrik will focus on marionette stuff these last days of this year

Mozmill Tests (Softvision)

Those are the bug fixed last week, we had 12:

• bug 1088638 Add automated test to install, disable, enable and remove services
• bug 1088561 Add automated test for plugins in different states
• bug 1104022 Create a page that loads insecure content while the page itself is secured
• bug 1093598 Move undo addon unistall test from testEnableDisableUndoUninstall.js to testAddons_uninstallExtension
• bug 1108990 Refactor testAdons_changeTheme from multiple files into a single file
• bug 1108993 Refactor testAdons_uninstallTheme from multiple files into a single file
• bug 1108994 Refactor testAddons_installMultipleExtensions from multiple files into a single file
• bug 1104794 Test failure 'The second item in the drop down list should be selected' in /testAwesomeBar/testAccessLocationBar.js
• bug 1106732 Test failure 'The correct SSL error title is shown - 'Unable to Connect Securely' should equal 'Secure Connection Failed in testSecurity/testSSLDisabledErrorPage.js
• bug 1025849 Create Mozmill Test for checking EV certificate status after a restart
• bug 1055453 Add new ui module for the page-info window
• bug 1113709 - Test failure "Plain-theme is marked to be enabled. - 'upgrade' should equal 'enable'" in testChangeTheme.js

TPS (Henrik)

• The signIn() failure is still present, looks like we have to wait for a fix from the DB vendor

WebRTC (Nils, Syd)

• TBD

Softvision Updates

• Detailed progress information from last week: https://groups.google.com/forum/#!topic/mozilla.dev.automation/8WbYuibZnk4

Personal Status

For the personal status please check the weekly status updates:

• 141222 - 141226

Meeting Notes

The meeting notes for this meeting can be read in the following etherpad:

• http://etherpad.mozilla.com:9000/automation-meeting-20141222

If you missed the meeting you can also watch it in our archive.

Action items

• [NEW] Cosmin - Work on a testcase for bug 1107405

Chair: Cosmin Notes: Barbara