Confirmed users
571
edits
QA/Automation/Meetings/141117: Difference between revisions
QA/Automation/Meetings/141117 (view source)
Revision as of 15:20, 18 November 2014
, 18 November 2014no edit summary
(Created page with "<small>previous meeting | Meetings | next meeting »</small> = Dial in = {...") |
AndreeaMatei (talk | contribs) No edit summary |
||
| (23 intermediate revisions by 7 users not shown) | |||
| Line 5: | Line 5: | ||
= General = | = General = | ||
* Attendees: | * Attendees: Andreea, Henrik, Teodor, Cosmin, Mihaela, Marc, Barbara, Nils | ||
* Chair: | * Chair: Henrik | ||
* Notes: | * Notes: Andreea | ||
= Last Weeks Action Items = | = Last Weeks Action Items = | ||
* {{new|}} [Henrik] follow up on bug 1083166 | * {{new|}} [Henrik] follow up on bug 1083166 | ||
** everything seems to work now, but we need to follow up here to get this investigated | |||
* {{done|}} [Henrik] Ask regarding open squid proxy | * {{done|}} [Henrik] Ask regarding open squid proxy | ||
** Proxy host has been created, now up to networking team to setup ACLs | ** Proxy host has been created, now up to networking team to setup ACLs | ||
** The VM has been setup, the requirements have been pushed to the team so we'll have this available soon | |||
** Once this has been landed, we can start to create tests for any network/proxy related testcases (we should check in-qa-testsuite) | |||
= Highlights = | = Highlights = | ||
* [ | * [Henrik] Firefox Automation report – week 37/38 2014: http://www.hskupin.info/2014/11/11/firefox-automation-report-week-37-38-2014/ | ||
** Henrik will add 2 new reports this week on his blog | |||
* [Henrik] Finally we removed all traces for ESR24 jobs from tests and CI (needs deployment to production) | |||
* [Henrik] Mozmill-CI staging got updates: https://github.com/mozilla/mozmill-ci/compare/ebe8b456d7f6...1ee11e3b78cc (Ubuntu 14.04, esr24 removal) | |||
** We should remove ubuntu 12.04 once we confirm 14.04 works fine | |||
* [Mihaela] Memory consumption increase in endurance tests starting November the 11th - {{bug|1099145}} Increased memory consumption starting Nov the 11th | |||
** all platforms affected, with a greater difference on Mac OS X 10.9.5 | |||
** could not reproduce locally so far, today Mihaela tried on 10.9 and 10.8, with fewer iterations and entities | |||
** will continue with 10 iterations/entities as we run on our CI, and maybe continue on staging if locally nothing happens | |||
** Henrik: maybe the team from AWSY have detected this yet? | |||
** Andrei: crosscheck if it's related to the updates we've recently done: java, flash, vmware | |||
** Henrik: maybe correlate with staging/production results, if we see this on both | |||
* Henrik landed the upgrade of Jenkins, details below on project updates | |||
= Blockers = | = Blockers = | ||
* List of blockers (http://bit.ly/Tph0NY): | * List of blockers (http://bit.ly/Tph0NY): | ||
** | ** [07/10] - Bug 735184 - RFE: Create Pulse notifications for update channel activities (maybe in Q4) | ||
** [09/01] - Bug 1038646 - Broken package dependencies when installing OpenJDK 7 on Ubuntu 12.04 machines (maybe wontfix because we want to get rid of it) | |||
*** Most likely we'll rely on openjdk, will be installed by default on linux | |||
*** We don't use java on our mozmill tests, so we don't need oracle versions | |||
** [09/03] - Bug 997230 - Setup Mac Mini for DeployStudio needed by PuppetAgain (needs action RelOps) | |||
** [10/29] - Bug 917204 - Add open squid proxy to qa.scl3.mozilla.com for testing purposes (waiting setting up network ACLs) | |||
** [10/29] - Bug 1066493 - TPS failure: Error: signIn() failed with: "error.sync.reason.serverMaintenance" (depends on bug 1057892) | |||
*** Waiting for the push on production for the database, hopefully by today/tomorrow will be landed and the failures will be fixed | |||
** [11/05] - Bug 1083166 - [SCL3] Pulse and maybe other connections are timing out in irregular intervals (maybe also causing test failures when loading web pages) | |||
** [11/07] - Bug 1094272 - [pulsetranslator] With SSL enabled publishing of normalized messages get stuck | |||
** [11/11] - Bug 1096367 - move aurora updates onto cdn temporarily | |||
*** We might need to find a solution here, it happened only once, in one day, but it might happen again | |||
= Contributor Spotlight = | = Contributor Spotlight = | ||
* Barbara Miller (:galgeek) will be our [ | * Barbara Miller (:galgeek) will be our [http://gnome.org/opw GNOME Outreach Program intern] from December to March | ||
** she spent some time on mozmill tests, she got more interested on automation | |||
** the program is trying to help people in groups underrepresented around the world | |||
= Roundtable = | = Roundtable = | ||
| Line 32: | Line 60: | ||
== Framework and Tools (Henrik) == | == Framework and Tools (Henrik) == | ||
=== Make Marionette ready for Mozmill tests === | === Make Marionette ready for Mozmill tests === | ||
* | * e10s support will most likely land for Firefox 37.0 by end of February 2015; we should have tests available when we go to Aurora at latest early Beta | ||
* Restart support landed for Marionette but we still miss the more important application triggered restart (see {{bug|1099127}}) | |||
* A couple more tests have been converted and examples created for feature checks, I will continue to do more myself this week | |||
** Will take a bit of time to be able to run mozmill tests on nightly, maybe try to do them for aurora and beta | |||
** Lots of work and enhancements for marionette for the ateam | |||
** David Burns was working on restart support, has been landed last week, we restart from inside the application, but stills represents only about 5% we actual need in our tests - blocking most of our important tests - like update tests | |||
=== Locale Rotation for Aurora builds in Mozmill-CI === | === Locale Rotation for Aurora builds in Mozmill-CI === | ||
* | * No updates | ||
=== Turn on PuppetAgain for Mozmill CI and TPS CI === | === Turn on PuppetAgain for Mozmill CI and TPS CI === | ||
* | * No updates | ||
=== Improvements to Testday bot [stretched] === | === Improvements to Testday bot [stretched] === | ||
* | * No updates | ||
== Test Automation (Softvision) == | == Test Automation (Softvision) == | ||
=== Creation of automated tests for important regressions === | === Creation of automated tests for important regressions === | ||
* | * New tests in work: | ||
** {{bug|1088561}} - Add automated test for plugins in different states | '''Almost done, only nits in last review.''' | |||
** {{bug|941477}} - Create a remote Mozmill test for the 'Get more Search Engines' feature | Waits for review | |||
** {{bug|1096209}} - Add automated tests for Firefox scrolling & zooming | Waits for review | |||
** {{bug|1088638}} - Add new automated test to install, disable, enable and remove services la new tests | 50% done | |||
= Project Updates = | = Project Updates = | ||
== Mozmill (Henrik) == | == Mozmill (Henrik) == | ||
* | * A couple of patches landed for Mozmill CI on staging (will be pushed to production this week) | ||
** There's no count of test failures in the emails from the CI list, this issue was disabled for a while | |||
** Enabling Ubuntu 14.04 nodes, and disabling 12.04 (which will be removed soon) | |||
** Removal of 24.0ESR related jobs | |||
** Upgrade to Jenkins 1.580.1: https://github.com/mozilla/mozmill-ci/issues/523 including fixes for: | |||
*** Huge HTTP session count over time | |||
*** Re-enabling Junit reports without wait times when collecting results | |||
*** Worked on the jenkins upgrade, huge patch, we need to enable it on staging | |||
*** Monitor plugin also had an upgrade, was tested locally and we will do more testing on staging | |||
** Fixed a problem with XShell, which was not able to kill jobs anymore | |||
** we weren't able to kill the process anymore, after the abort on CI happened | |||
* Fixed Flashplugin-installer issue for download notification windows ({{bug|949427}}) | |||
* Problem on how we're connecting our slave nodes - javaws. There's an issue on jenkins for linux when the node got disconnected. Henrik looked over multiple solutions, filed more issues against jenkins | |||
** wrote a little script that can be used for puppet too | |||
== Mozmill Tests (Softvision) == | == Mozmill Tests (Softvision) == | ||
* | * Fixed: | ||
** {{bug|1060254}} - Test failure 'Geolocation position is: unavailable' in /testGeolocation/testShareLocation.js | |||
** {{bug|678456}} - getSuggestions in search.js returns suggestions for the first letter only | |||
** {{bug|994102}} - Search engines don't work right away after installation | |||
** {{bug|1087202}} - Test failure "Element.waitForElement(): Element 'Selector: *[label='Test Bookmark 1']'" in /testBookmarks_OpenAndClose/test1.js | |||
** {{bug|987565}} - Test failure "controller.waitForPageLoad" in testBookmarks_OpenAndClose/test1.js and testBookmarks_OpenAllInTabs/test1.js | |||
** {{bug|1097700}} - Test failure 'Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [nsIFile.remove]' in /testAddons/testInstallUninstallSoftBlocklistedExtension.js | |||
** {{bug|689417}} - tests/functional/testToolbar/testStopReloadButtons.js doesn't fail when explicitly broken // just closed, this was previously fixed by {{bug|954156}} | |||
** {{bug|1097682}} - Test failure 'Unknown property - identity.mixed_display_loaded' in /testSecurity/testMixedContentPage.js | |||
** {{bug|1084333}} - Add some more methods and a property to BaseWindow class | |||
** {{bug|1079725}} - Add a BaseInContentPage class | |||
** {{bug|1097095}} - Windows XP End of Support Pop-up on all XP production machines | |||
* WFM: | |||
** {{bug|929357}} - Timeout failure in between functional testruns | |||
** 11 fixed bugs last week and another 7 are in progress | |||
== TPS (Henrik) == | == TPS (Henrik) == | ||
* | * Needs update for pulse script to support Pulse Guardian | ||
** upgrade the pulse script to pulse gardian | |||
* Hopefully this week the server database will be updated, so the signIn() problem might be gone | |||
** we still have test failures, but are related to server problems - only related to users, dynamically create accounts on the servers, because on the machine we run tests all are passing | |||
* Shall we permanently enable the new CI? Current failures are all most likely related to server-side issues | |||
** Cosmin: we don't get notifications for failures from the old system - this has to be checked | |||
** Marc: as long as we're not missing something, I'm good with doing the switch | |||
== WebRTC (Nils, Syd) == | == WebRTC (Nils, Syd) == | ||
* | * Started to use the lab for loop, there's work in progress to be able to run tests, due to missing infrastructure for these different components | ||
* monitoring failures, might be more infrastructure related than webrtc code | |||
= Softvision Updates = | = Softvision Updates = | ||
* Detailed progress information from last week: https://groups.google.com/forum/#!topic/mozilla.dev.automation/ | * Detailed progress information from last week: https://groups.google.com/forum/#!topic/mozilla.dev.automation/xpMjUTsacdo | ||
= Personal Status = | = Personal Status = | ||
| Line 74: | Line 145: | ||
==Action items== | ==Action items== | ||
* {{new|}} | * {{ok|}} [Henrik] follow up on bug 1083166 | ||
* {{new|}} [Andreea] Check in-qa-testsuite flags for proxy/network tests | |||
* {{new|}} [Mihaela] Check with the team from AWSY if they have detected our endurance memory increased consumption | |||
* {{new|}} [Henrik] goals wiki page | |||
* {{new|}} [Henrik] check if we can shutdown the old tps system | |||
* Chair: Andrei | |||
* Notes: Teodor | |||
[[Category:Wikipage templates]] | [[Category:Wikipage templates]] | ||