Auto-tools/Automation Development/Meetings/130819: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
 
(14 intermediate revisions by 7 users not shown)
Line 17: Line 17:
* [dave] performance tests npw being run on the 'leo' device for [https://datazilla.mozilla.org/b2g/?branch=v1-train&device=d300&range=7&test=cold_load_time cold load time] and [https://datazilla.mozilla.org/b2g/?branch=v1-train&device=d300&range=7&test=fps fps].
* [dave] performance tests npw being run on the 'leo' device for [https://datazilla.mozilla.org/b2g/?branch=v1-train&device=d300&range=7&test=cold_load_time cold load time] and [https://datazilla.mozilla.org/b2g/?branch=v1-train&device=d300&range=7&test=fps fps].
* [henrik] For all mozmill-ci nodes the update checks have been disabled for all applications and the OS itself (see bug 900860)
* [henrik] For all mozmill-ci nodes the update checks have been disabled for all applications and the OS itself (see bug 900860)
* [henrik] Most likely this week we will be able to test different locales for beta and release candidates (needs my patch for pulsetranslator ({{bug|857980}}) pushed to production only)
* You can find all blockers here: http://bit.ly/Tph0NY
* You can find all blockers here: http://bit.ly/Tph0NY
** <strike>Bug 904193 - Increase the memory of mm-ci-master.qa.scl3.mozilla.com to 4GB</strike>
** <strike>Bug 904193 - Increase the memory of mm-ci-master.qa.scl3.mozilla.com to 4GB</strike>
Line 24: Line 25:
* [henrik] Branching mozmill repository for mozmill-2.0 release
* [henrik] Branching mozmill repository for mozmill-2.0 release
** Given that we don't want any new features but only bug fixes now for Mozmill 2.0 we should branch into a new hotfix-2.0 branch, so development of new features doesn't stop
** Given that we don't want any new features but only bug fixes now for Mozmill 2.0 we should branch into a new hotfix-2.0 branch, so development of new features doesn't stop
* [dave] Could SoftVision help with merges?
* [dave] Where should the Mozmill CI troubleshooting document live?
* [henrik] QA Show and Tell demonstation
** No more suggestions as from Dave? https://groups.google.com/forum/#!topic/mozilla.dev.automation/YXw_C0Sx4XA
* [Andrei] testrun report:
** Lets clarify when it should be done. Beta week 1 and 6 - 1 time, after beta runs, Beta week 2-4 - 2 times after each run Tuesday and Thursday.


= Goals Overview =
= Goals Overview =
Line 44: Line 51:


= Softvision Updates =
= Softvision Updates =
== Desktop Automation ==
* Goals progress: Mozmill 2.0 releases
* Fixed:
**
* Working on:
** Andreea
*** {{bug|888924}} - Test failure "Memory cache has no entries after PB mode - 'undefined' should equal '0'" in /testPrivateBrowsing/testAboutCache.js - testing fix
*** {{bug|905988}} - [ff] Test failure "Search bar has the focus" in /testSearch/testSearchViaShortcut.js
*** {{bug|758187}} - Write a new mozmill test for checking location sharing - update patch
*** {{bug|795398}} - Mozmill test to check that MD5 hash signatures are no longer accepted - the site not getting verified it seems to be an issue with the certificate being ready
** Cosmin
*** {{bug|879950}} - Remove Lookup methods and Xpath elements from lib/tabs.js
*** {{bug|890181}} - Test failure "Tab has been closed" when trying to close a tab with close button
*** {{bug|894883}} - Failure "The Add-ons Manager has been opened in a second tab - '1' should equal '2'' in /testAddons/testManagerKeyboardShortcut.js
*** {{bug|905960}} - [es-CL] Test failure "Owner location string is equal to the Cert location" in /testSecurity/testGreenLarry.js
** Mario
*** {{bug|840022}} - Test failure "The forward button has been made visible for the 1 page" in testToolbars/testBackForwardButtons.js
*** {{bug|905033}} - Test failure "not-a-web-forgery report page is loaded" in /testSafeBrowsingNotificationBar.js
*** {{bug|906599}} - Test failure "controller(): Window could not be initialized" in /ide@seleniumhq.org/tests/testCommands/
*** {{bug|895843}} - [id] testReorderSearchEngines.js fails with 'Modal dialog has been found and processed'
*** {{bug|905942}} - [km] Test failure "The all tabs popup should have been opened" in /testBackgroundTabScrolling.js
** Andrei
*** {{bug|905170}} - Test failure "The theme 'plain.theme@quality.mozilla.org' is enabled - got 'false'" in testAddons_changeTheme/test2.js
*** {{bug|900911}} - Test failure 'Active tab url should equal the page url' in /testAwesomeBar/testSwitchToTab.js
{| border="1" cellpadding="2" align="center"
|+<b> Waiting for review/Feedback/Need info</b>
| <b>Bug number</b> || <b>Status</b>  || <b>Requestee</b>  || <b>Requester</b>  || <b>Bug Subject</b>
|-
| {{bug|489474}} || Review || Henrik || Andreea || Write automated tests for user generated video controls
|-
| {{bug|800872}} || Review || Henrik || Andreea || Test failure "Window number '1' has been opened" in endurance/testTabbedBrowsing_OpenNewWindow/test1.js
|-
| {{bug|786306}} || Review || Henrik || Mario || Restart tests logic
|-
| {{bug|838192}} [[https://github.com/mozilla/mozmill-dashboard/pull/50 Pull Request]] || Review || Henrik/Dave || Andrei || Add screenshot functionality to assertion module
|-
| {{bug|599290}} || Feedback || Henrik || Cosmin || Update SoftwareUpdateAPI to support complete relocation to the about dialog
|-
| {{bug|879953}} || Review || Andreea || Cosmin || Remove Lookup methods and Xpath elements from lib/toolbars.js
|-
| {{bug|880426}} || Review || Dave/Henrik || Andrei || Enhance Mozelement with touch events
|-
| {{bug|900911}} || Review || Dave/Henrik/Andreea || Andrei || Test failure 'Active tab url should equal the page url' in /testAwesomeBar/testSwitchToTab.js
|-
| {{bug|886811}} || Review || Dave/Henrik || Andrei || Use controller.restartApplication() or controller.stopApplication() instead of controller.startUserShutdown() for mozmill-2.0
|-
| {{bug|898378 || Review || Henrik/Dave || Andrei || Mutt test failure "keypress() with pressed shift key succeeded. - '' should equal 'F'" in "mutt/mutt/tests/js/controller/synthesize_events.js"
|}


== Mobile (Robocop) ==
== Mobile (Robocop) ==
*  
* Progress on [https://docs.google.com/spreadsheet/ccc?key=0AhE7m4JB2j6tdDJBT2dlbVJwUk9PSy1RbHo4WVNiUGc#gid=8 Robocop Automation]
* Bugs with work in progress:
** {{Bug|820859}} - Robocop: Add test for 'Clear Private Data'
** {{Bug|830834}} - Robocop: Add test for 'Show Search Suggestions' feature
** {{Bug|836341}} - Robocop: Add test for "Clear download history"
** {{Bug|854043}} - Intermittent testWebContentContextMenu | Verify that the download started - The download started
** {{Bug|878587}} - Intermittent testMasterPassword | Exception caught - junit.framework.AssertionFailedError: Click can not be completed!
** {{Bug|900664}} - Robocop testImportFromAndroid fails on x86 emulator / RAZR
** {{Bug|899187}} - [fig] Create new testBookmarksPage test to replace original testBookmarksTab.java.in
* Bugs with patches awaiting review:
** {{Bug|896574}} - [fig] Fix and re-enable testAddSearchEngine
** {{Bug|899182}} - [fig] Re-implement robocop testBookmark.java.in
* Bugs awaiting integration:
** {{Bug|906662}} - Fix openAboutHomeTab to work with the changes to the tablet about:home UI
** {{Bug|896566}} - [fig] Fix and re-enable testClearPrivateData
** {{Bug|888277}} - Robocop: Add test for Private Browsing
* Bugs recently integrated:
** {{Bug|896557}} - [fig] Fix and re-enable testWebContentContextMenu
** {{Bug|901432}} - [FIG] Cleanup BaseTest and create a new Utility class to organize the methods in BaseTest
** {{Bug|901445}} - [FIG] Add the clickLongOnScreen implementation from robotium 4.0 to Robocop
** {{Bug|869277}} - Intermittent testAddSearchEngine | The number of search results has increased - got 5, expected 10
* '''Issues'''


= Personal Status =
= Personal Status =
Line 57: Line 139:


==Action items==
==Action items==
* {{new|}} %name%: %task%
* {{new|}} [henrik] Create a hotfix-2.0 branch on mozmill
* {{new|}} [dave] Create a Mozmill CI troubleshooting guide on the github wiki
* {{new|}} [andrei] talk to Anthony and Juan about what would help him with Release and Beta runs and see how we can help with that (in relation with a tesrun Report)
* {{new|}} [andreea & andrie] create the merge patches for the next Firefox release (September 10th & 17th)
* {{new|}} [henrik] Update the goals pages on the wiki

Latest revision as of 14:13, 20 August 2013

previous meeting | Meetings | next meeting »

Dial in

 # When:     Every Monday at 8:45am PDT/PST
 # Vidyo:    https://v.mozilla.com/flex.html?roomdirect.html&key=PGtLpx3XQGJz
 # Phone:    650-903-0800 or 650-215-1282 x92 Conf# 9654 (US/INTL)
 #           1-800-707-2533 (pin 369) Conf# 9654 (US)
 # IRC:      irc://irc.mozilla.org:6697/#automation
 # IRC logs: http://irclog.gr/#browse/irc.mozilla.org/automation

General

  • Attendees:
    • TBD
  • Chair: Rob
  • Notes: Dave

Last Weeks Action Items

  • None

Highlights

  • [henrik] Andrei Eftimie is our new peer for mozmill-tests and mozmill-automation
  • [dave] performance tests npw being run on the 'leo' device for cold load time and fps.
  • [henrik] For all mozmill-ci nodes the update checks have been disabled for all applications and the OS itself (see bug 900860)
  • [henrik] Most likely this week we will be able to test different locales for beta and release candidates (needs my patch for pulsetranslator (bug 857980) pushed to production only)
  • You can find all blockers here: http://bit.ly/Tph0NY
    • Bug 904193 - Increase the memory of mm-ci-master.qa.scl3.mozilla.com to 4GB
    • Bug 905623 - No Aurora 25.0a2 64bit autoupdate since around August 12th

Roundtable

  • [henrik] Branching mozmill repository for mozmill-2.0 release
    • Given that we don't want any new features but only bug fixes now for Mozmill 2.0 we should branch into a new hotfix-2.0 branch, so development of new features doesn't stop
  • [dave] Could SoftVision help with merges?
  • [dave] Where should the Mozmill CI troubleshooting document live?
  • [henrik] QA Show and Tell demonstation
  • [Andrei] testrun report:
    • Lets clarify when it should be done. Beta week 1 and 6 - 1 time, after beta runs, Beta week 2-4 - 2 times after each run Tuesday and Thursday.

Goals Overview

Project Updates

Gaia UI (Dave)

  • [rwood] Working with Dave on merging the gaia-ui endurance tests into the main /gaia repo

WebAPI (Rob)

  • [rwood] No updates

WebRTC (Henrik)

  • Goals progress:
  • TBD

Games Benchmarking (Rob, Joel, BC)

  • [rwood] Working on getting Octane running on Leo; benchmarking tests for Firefox OS to be run on 512MB+ devices only as Inari, Unagi don't have enough device memory (256MB)

Softvision Updates

Desktop Automation

  • Goals progress: Mozmill 2.0 releases
  • Fixed:
  • Working on:
    • Andreea
      • bug 888924 - Test failure "Memory cache has no entries after PB mode - 'undefined' should equal '0'" in /testPrivateBrowsing/testAboutCache.js - testing fix
      • bug 905988 - [ff] Test failure "Search bar has the focus" in /testSearch/testSearchViaShortcut.js
      • bug 758187 - Write a new mozmill test for checking location sharing - update patch
      • bug 795398 - Mozmill test to check that MD5 hash signatures are no longer accepted - the site not getting verified it seems to be an issue with the certificate being ready
    • Cosmin
      • bug 879950 - Remove Lookup methods and Xpath elements from lib/tabs.js
      • bug 890181 - Test failure "Tab has been closed" when trying to close a tab with close button
      • bug 894883 - Failure "The Add-ons Manager has been opened in a second tab - '1' should equal '2 in /testAddons/testManagerKeyboardShortcut.js
      • bug 905960 - [es-CL] Test failure "Owner location string is equal to the Cert location" in /testSecurity/testGreenLarry.js
    • Mario
      • bug 840022 - Test failure "The forward button has been made visible for the 1 page" in testToolbars/testBackForwardButtons.js
      • bug 905033 - Test failure "not-a-web-forgery report page is loaded" in /testSafeBrowsingNotificationBar.js
      • bug 906599 - Test failure "controller(): Window could not be initialized" in /ide@seleniumhq.org/tests/testCommands/
      • bug 895843 - [id] testReorderSearchEngines.js fails with 'Modal dialog has been found and processed'
      • bug 905942 - [km] Test failure "The all tabs popup should have been opened" in /testBackgroundTabScrolling.js
    • Andrei
      • bug 905170 - Test failure "The theme 'plain.theme@quality.mozilla.org' is enabled - got 'false'" in testAddons_changeTheme/test2.js
      • bug 900911 - Test failure 'Active tab url should equal the page url' in /testAwesomeBar/testSwitchToTab.js
Waiting for review/Feedback/Need info
Bug number Status Requestee Requester Bug Subject
bug 489474 Review Henrik Andreea Write automated tests for user generated video controls
bug 800872 Review Henrik Andreea Test failure "Window number '1' has been opened" in endurance/testTabbedBrowsing_OpenNewWindow/test1.js
bug 786306 Review Henrik Mario Restart tests logic
bug 838192 [Pull Request] Review Henrik/Dave Andrei Add screenshot functionality to assertion module
bug 599290 Feedback Henrik Cosmin Update SoftwareUpdateAPI to support complete relocation to the about dialog
bug 879953 Review Andreea Cosmin Remove Lookup methods and Xpath elements from lib/toolbars.js
bug 880426 Review Dave/Henrik Andrei Enhance Mozelement with touch events
bug 900911 Review Dave/Henrik/Andreea Andrei Test failure 'Active tab url should equal the page url' in /testAwesomeBar/testSwitchToTab.js
bug 886811 Review Dave/Henrik Andrei Use controller.restartApplication() or controller.stopApplication() instead of controller.startUserShutdown() for mozmill-2.0
898378 Review Henrik/Dave Andrei Mutt test failure "keypress() with pressed shift key succeeded. - should equal 'F'" in "mutt/mutt/tests/js/controller/synthesize_events.js"

Mobile (Robocop)

  • Progress on Robocop Automation
  • Bugs with work in progress:
    • bug 820859 - Robocop: Add test for 'Clear Private Data'
    • bug 830834 - Robocop: Add test for 'Show Search Suggestions' feature
    • bug 836341 - Robocop: Add test for "Clear download history"
    • bug 854043 - Intermittent testWebContentContextMenu | Verify that the download started - The download started
    • bug 878587 - Intermittent testMasterPassword | Exception caught - junit.framework.AssertionFailedError: Click can not be completed!
    • bug 900664 - Robocop testImportFromAndroid fails on x86 emulator / RAZR
    • bug 899187 - [fig] Create new testBookmarksPage test to replace original testBookmarksTab.java.in
  • Bugs with patches awaiting review:
    • bug 896574 - [fig] Fix and re-enable testAddSearchEngine
    • bug 899182 - [fig] Re-implement robocop testBookmark.java.in
  • Bugs awaiting integration:
    • bug 906662 - Fix openAboutHomeTab to work with the changes to the tablet about:home UI
    • bug 896566 - [fig] Fix and re-enable testClearPrivateData
    • bug 888277 - Robocop: Add test for Private Browsing
  • Bugs recently integrated:
    • bug 896557 - [fig] Fix and re-enable testWebContentContextMenu
    • bug 901432 - [FIG] Cleanup BaseTest and create a new Utility class to organize the methods in BaseTest
    • bug 901445 - [FIG] Add the clickLongOnScreen implementation from robotium 4.0 to Robocop
    • bug 869277 - Intermittent testAddSearchEngine | The number of search results has increased - got 5, expected 10
  • Issues

Personal Status

For the personal status please check the weekly status updates:

Meeting Notes

The meeting notes for this meeting can be read in the following etherpad:

Action items

  • [NEW] [henrik] Create a hotfix-2.0 branch on mozmill
  • [NEW] [dave] Create a Mozmill CI troubleshooting guide on the github wiki
  • [NEW] [andrei] talk to Anthony and Juan about what would help him with Release and Beta runs and see how we can help with that (in relation with a tesrun Report)
  • [NEW] [andreea & andrie] create the merge patches for the next Firefox release (September 10th & 17th)
  • [NEW] [henrik] Update the goals pages on the wiki
Retrieved from "https://wiki.allizom.org/index.php?title=Auto-tools/Automation_Development/Meetings/130819&oldid=697098"