QA/Automation/Meetings/140407

previous meeting | Meetings | next meeting »

Dial in

 # When:     Every Monday at 8:45am PDT/PST
 # Vidyo:    https://v.mozilla.com/flex.html?roomdirect.html&key=PGtLpx3XQGJz
 # Phone:    650-903-0800 or 650-215-1282 x92 Conf# 9654 (US/INTL)
 #           1-800-707-2533 (pin 369) Conf# 9654 (US)
 # IRC:      irc://irc.mozilla.org:6697/#automation
 # IRC logs: http://irclog.gr/#browse/irc.mozilla.org/automation

General

• Attendees: Andrei, Andreea, Mihaela, Daniel, Henrik, Cosmin, Nils, Otilia
• Chair: Daniel
• Notes: Andreea

Last Weeks Action Items

• [DONE] [Henrik] Update the goal list and send and email update
• [DONE] [Nils] Check if links to WebRTC bugs made it into the Wiki from last week
• [DONE] [Daniel] file an issue against mozmill-dashboard to implement a testrun status report
  • https://github.com/mozilla/mozmill-dashboard/issues/93
• [CARRY OVER] [Mihaela, Daniel] Get a list of the tests needed to automate - create an email thread on the automation list to get them prioritized (in-qa-testsuite? flag)
  • https://etherpad.mozilla.org/tests-to-automate
  • once final list will be over, needed bugs will be filed and the email thread will be created
• [CARRY OVER] [Softvision] Metro
  • finish mozmill tests for metro
    • bug 883860 - Create metro mozmill test for Pop-up blocker
    • Only this test left to be completed;
  • bug 968079 - Update tabs.js metro library to use UI elements only
    • final review provided and it's almost finished

Highlights

• [henrik] Mozmill Automation 2.0.6.1 released: https://groups.google.com/forum/#!topic/mozilla.dev.automation/ubnyQviFwao
• New blockers, any updates?
  • bug 992154 - Cronjob for testcase-data on mozqa1.qa.scl3 no longer updates HTTP root
  • bug 992753 - Update SSL certificate for ssl-md5.mozqa.com
• Github blockers
  • https://github.com/mozilla/mozmill-ci/issues/358 - Cannot delete workspace problem on Windows.
• You can find all blockers here: http://bit.ly/Tph0NY

Roundtable

• New goals for Q2: https://wiki.mozilla.org/QA/Goals/2014q2

Goals Overview

• https://wiki.mozilla.org/QA/Automation#Goals

Project Updates

Mozmill (Henrik)

• Mozmill Automation 2.0.6.1 released: https://groups.google.com/forum/#!topic/mozilla.dev.automation/ubnyQviFwao
• With Mozmill-CI we will be able to handle RC->Beta updates soon. We need a way to test this
• Henrik detected an issue with mozmill-env. It misses installed setuptools. An upgrade of virtualenv will fix this, so we have to recreate the environments (https://github.com/mozilla/mozmill-environment/issues/75) before we can upgrade to Python 2.7.5 on the remaining boxes

TPS (Henrik)

• All remaining backports have been landed on mozilla-aurora
• Bug 986190 - [tps] Synced data on the server is not always wiped at the end of a test
• Bug 990797 - TPS CI errors out when trying to e-mail results
• Bug 989045 - [tps] Registering of observer topics happens too late so 'weave:engine:start-tracking' can be missed during startup
• Bug 989249 - test_addon_reconsiling.js broken due to missing sync
• Work happens on bug 992198, where we want to allow wiping the remote server on first sync

WebRTC (Nils)

• Solved the highest WebRTC orange regarding missing canplaythrough event on B2G last week https://bugzilla.mozilla.org/show_bug.cgi?id=989047
• WebRTC tiger team started to setup an ideal WebRTC calling environment with dedicated machines and network connection
• WebRTC tiger team solved the problem of errors in IPC tests are not propagating properly up to mochitest
• Found 2 issues with RTCPeerConnection.close(), one of them is solved already (https://bugzilla.mozilla.org/show_bug.cgi?id=991368) the other still being discussed (https://bugzilla.mozilla.org/show_bug.cgi?id=989936)

Softvision Updates

Desktop Automation

• Goals progress:
• Updates: A week full of regressions and crashes, mostly on endurance and security tests and crashes related to Flash.

• Resolved Fixed:
  • [Mozmill automation] Added --override-update-settings parameter for update testruns (#132) #133 https://github.com/mozilla/mozmill-automation/pull/133
  • [Mozdownload] Check if self.timestamp in method build_filename is really of date type and not just a string (#196) #197 https://github.com/mozilla/mozdownload/pull/197
  • bug 988867 - Enhance search engines so they will give mockup suggestions
• Working on:
  • Andreea
    • bug 991736 - Disable endurance flash tests on esr24 for Windows due to crashes
    • bug 711129 - Mozmill Endurance test for Open & Close a popup window
    • bug 990557 - Test failure "[object Object] - got 'false' " in testSecurity/testMD5HashSignature.js
  • Andrei
    • bug 891288 - Create subdomains of mozqa with self-signed SSL certificates for all supported versions versions (SSL3 - TLS 1.1)
    • bug 989922 - Test failure "Element.waitForElement(): Element 'ID: errorTitleText' has been found" in testSecurity/testSSLDisabledErrorPage.js
    • bug 972912 - Mozmill fails to ran all expected tests
    • bug 990509 - TPS test failure "[phase3] Exception caught: ASSERTION FAILED! error locating tab" in test_tabs.js
  • Cosmin
    • [Mozmill CI] - Added --override-update-settings parameter for ondemand_update job (#428) #429 https://github.com/mozilla/mozmill-ci/pull/429
    • bug 911257 - [mozrunner] Add feature to set MetroFirefox as the default browser
    • bug 904577 - Method exists() from elementslib.Elem returns true even after the node has been removed
    • bug 732353 - Disable all Discovery Pane tests due to unpredictable web dependencies
    • bug 942737 - Test failure "Suggestions from two search engines are available - '1' should equal '2' " in testSearchSuggestions.js
    • bug 799149 - Add mozmill test to ensure that there are no conflicts in context menu access keys.
    • bug 992139 - [mozversion] Enhance mozversion to return "application_name" if "application_code_name" doesn't exists
  • Daniel
    • bug 987098 - Failure 'Argument 1 of Window.getComputedStyle is not an object' in testPopups\testPopupsBlocked.js
    • bug 969428 - Create form data backend library for handling form related actions
    • bug 936478 - Test failure "controller.waitForPageLoad(): Timeout waiting for page loaded" in /testSecurity/testSafeBrowsingWarningPages.js
    • bug 883860 - Create metro mozmill test for Pop-up blocker
    • bug 968079 - Update tabs.js metro library to use UI elements only
    • bug 816084 - Update toolbars.js shared module to move elements from security tests in locationBar.getElement()
    • bug 863139 - Add ui module for in-content security pages & security dialogs
    • bug 991066 - Test failure "Video data is available" in endurance/testVideo_OGVBuffering/test1.js
  • Mihaela
    • bug 974914 - Add test for the functionality of Panel Menu buttons in Australis
      • failures investigations and new fixes
      • partial patch green in tbpl
    • bug 973641 - Add test for the functionality of addon button in Australis
      • failures investigation - blocked by bug 985332
    • Moztrap need_automation tests - triage and check for existing coverage
    • work at a list of tests needed to automate for in-testsuite/in-qa-testsuite bugs

Personal Status

For the personal status please check the weekly status updates:

• 140407 - 140414

Meeting Notes

The meeting notes for this meeting can be read in the following etherpad:

• http://etherpad.mozilla.com:9000/automation-meeting-20140407

If you missed the meeting you can also watch it in our archive.

Action items

• [CARRY OVER] [Mihaela, Daniel] Get a list of the tests needed to automate - create an email thread on the automation list to get them prioritized (in-qa-testsuite? flag)
• [CARRY OVER] [Softvision] Metro
  • finish mozmill tests for metro
    • bug 883860 - Create metro mozmill test for Pop-up blocker
    • bug 968079 - Update tabs.js metro library to use UI elements only
• [NEW] [Henrik] To update the goals for the metric - 10 tests to automate (Kamil is also helping with the tests)
• [NEW] [Henrik] Talk to Marc about the goals for Nils