Auto-tools/Automation Development/Meetings/130715: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
 
(26 intermediate revisions by 9 users not shown)
Line 5: Line 5:


= General =
= General =
* Attendees:  
* Attendees:
** TBD
** Dave, Andrei, Daniela, Andreea, Rob, Adrian, Henrik, Cosmin, Otilia
* Chair: TBD
* Chair: Henrik
* Notes: TBD
* Notes: Andreea


= Last Weeks Action Items =
= Last Weeks Action Items =
Line 17: Line 17:


= Highlights =
= Highlights =
* [%name%] %highlight%
* [henrik] Due to the constant failures in submitting Mozmill test results to iriscouch we have been moved to a new server. Big thanks goes to Jason Smith from IrisCouch for his ultra-fast response and action!
* [henrik] About to fix a major bug in jsbridge which leds to dataloss when sending too much data. See bug 764640.
* [dave] [https://travis-ci.org/mozilla/gaia-ui-tests/builds/8776316 First passing Travis-CI build] for Gaia UI tests on desktop build!
* [dave] HTML report for Gaia UI tests [http://qa-selenium.mv.mozilla.com:8080/view/B2G%20Unagi/job/b2g.unagi.mozril.gaia.master.ui/101/HTML_Report/? finally available]!
* [dave] Gaia UI tests are now running (and passing!) [https://tbpl.mozilla.org/?tree=Cedar in TBPL against the Cedar branch]
* [dave] b2gpopulate 0.8 released with support for call history and MMS messages
* [dave] b2gperf 0.5 released
* New blockers, any updates?
* New blockers, any updates?
** %new blockers%
** {{bug|892928}} - b2gperf is failing to gather FPS measurements on mozilla-central builds
** {{bug|893693}} - TypeError: IccHelper.removeEventListener is not a function
* You can find all blockers here: http://bit.ly/Tph0NY
* You can find all blockers here: http://bit.ly/Tph0NY


Line 27: Line 34:
** P2 - mozmill 2.0 blocker (if release is approaching, this can take higher priority in order to finish in time)
** P2 - mozmill 2.0 blocker (if release is approaching, this can take higher priority in order to finish in time)
** P3 - not that often failures, medium enhancements
** P3 - not that often failures, medium enhancements
** P4 - new tests - QA needed have priority
** P4 - new tests - the ones needed by the QA team have higher priority
** P5 - refactoring, small enhancements
** P5 - refactoring, small enhancements
** Dependecies take the blocking bug's priority
** Dependecies take the blocking bug's priority
*** Our prioritized backlog: https://docs.google.com/spreadsheet/ccc?key=0AgbJACdAek5ndGw5TWNuWWw1VFFzelZHektWRTNaSnc#gid=10
*** Our prioritized backlog: https://docs.google.com/spreadsheet/ccc?key=0AgbJACdAek5ndGw5TWNuWWw1VFFzelZHektWRTNaSnc#gid=10
* [Mario] UI objects for content pages
** Do we want to create UI objects in the libraries for content pages?
*** Related to Bug 835383 : https://bugzilla.mozilla.org/show_bug.cgi?id=835383
* [henrik] Do we want to make use of github releases instead of uploading versions to pypi directly?
** See https://github.com/blog/1547-release-your-software
** That would simplify the release process drastically for our projects
** Downside would be when github is not reachable, you wont be able to install applications


= Goals Overview =
= Goals Overview =
Line 39: Line 55:
== Gaia UI (Dave) ==
== Gaia UI (Dave) ==
* Goals progress:
* Goals progress:
* TBD
* See highlights


== WebAPI (Rob) ==
== WebAPI (Rob) ==
* Goals progress:
* Submitted patch to fix two WebTelephony WebAPI failures
* TBD


== WebRTC (Henrik) ==
== WebRTC (Henrik) ==
* Goals progress:
* In process to get the datachannel framework backported - currently blocked by try outage
* TBD
* Most likely working on two or three other enhancements
 
== Games Benchmarking (Rob, Joel, BC) ==
* [rwood] Working on getting speedtests running on Firefox OS


= Softvision Updates =
= Softvision Updates =
Line 61: Line 79:
** {{bug|891842}} - Test failure 'Expression "{"anonid":"findbar-textbox"}" returned null. Anonymous == true' in testFindInPage.js  
** {{bug|891842}} - Test failure 'Expression "{"anonid":"findbar-textbox"}" returned null. Anonymous == true' in testFindInPage.js  
** {{bug|892946}} - Test failure in testFindInPage due to the fact that the find bar was moved to the top
** {{bug|892946}} - Test failure in testFindInPage due to the fact that the find bar was moved to the top
** {{bug|892388}} - Create a clone of restartless.xpi for uploading to AMO with an end user license agreement
** {{bug|763461}} - Mozmill tests for cut-off elements should not report failure for scrollable direction


* Working on:
* Working on:
Line 66: Line 86:
*** {{bug|885221}} - controller.waitForPageLoad() failures with mozmill 2.0 due to closeAllTabs method opening about:newtab instead of about:blank
*** {{bug|885221}} - controller.waitForPageLoad() failures with mozmill 2.0 due to closeAllTabs method opening about:newtab instead of about:blank
*** {{bug|880135}} - Test failure "The tab with index '1' has been selected" in testOpenInBackground.js
*** {{bug|880135}} - Test failure "The tab with index '1' has been selected" in testOpenInBackground.js
*** {{bug|684801}} - Timeout failure in /testTabView_SwitchTabs/test1.js | TabView is still open.
** Andreea
** Andreea
*** {{bug|795398}} - Mozmill test to check that MD5 hash signatures are no longer accepted
*** {{bug|795398}} - Mozmill test to check that MD5 hash signatures are no longer accepted
*** {{bug|880417}} - Create user interface shared module for metro
*** {{bug|880417}} - Create user interface shared module for metro
*** {{bug|853005}} - Update automation script should exit with a non-zero exit code if unable to get update channel
*** {{bug|853005}} - Update automation script should exit with a non-zero exit code if unable to get update channel
** Mario
*** {{bug|874394}} - [Australis] Test failure in '/testAwesomeBar/testPasteLocationBar.js' due to the urlbar-container not being found
*** {{bug|835296}} - Add test that installs an addon with EULA directly from addons.mozilla.org
*** {{bug|835383}} - Add test that installs an addon without EULA directly from addons.mozilla.org
*** {{bug|835132}} - Test failure "Current domain name matches target domain name" in /testSearch/testSearchSelection.js
** Andrei
*** {{bug|888196}} - Memory increase in endurance testsruns on Win 8 (since June the 16th) - running more endurance tests on SCL


{| border="1" cellpadding="2" align="center"
{| border="1" cellpadding="2" align="center"
Line 80: Line 108:
|-
|-
| {{bug|786306}} || Review || Henrik || Mario || Restart tests logic
| {{bug|786306}} || Review || Henrik || Mario || Restart tests logic
|-
| {{bug|889314 }} || Review || Dave || Mario || Update the restartless.xpi addon to be compatible with AMO. Set maxValue to a valid value instead of "*"
|-
| {{bug|835383}} || Feedback || Andreea || Mario || Add test that installs an addon without EULA directly from addons.mozilla.org
|-
| {{bug|880135}} || Need info || Henrik || Daniela || Test failure "The tab with index '1' has been selected" in testOpenInBackground.js
|-
|-
| {{bug|732353}} || Feedback || Henrik || Daniela ||  Disable all Discovery Pane tests due to unpredictable web dependencies
| {{bug|732353}} || Feedback || Henrik || Daniela ||  Disable all Discovery Pane tests due to unpredictable web dependencies
Line 94: Line 116:
|-
|-
| {{bug|871441}} || Feedback || Henrik || Daniela || Focus inside the select method is not set properly on the menulist
| {{bug|871441}} || Feedback || Henrik || Daniela || Focus inside the select method is not set properly on the menulist
|-
| {{bug|888853}} || Review || Henrik || Andrei || Enhance controller.restartApllication() to handle custom restart flags
|-
|-
| {{bug|886811}} || Review || Henrik || Andrei || Use controller.restartAppplication() instead of controller.startUserShutdown() for mozmill-2.0
| {{bug|886811}} || Review || Henrik || Andrei || Use controller.restartAppplication() instead of controller.startUserShutdown() for mozmill-2.0
|-
| {{bug|885723}} || Needinfo || Mano || Andrei || Test failure "Autocomplete popup has been opened" in testSwitchToTab.js
|-
| {{bug|887315}} || Review || Henrik || Andrei || Mutt test window_focus.js fails with: "Windows have maintained their order - '[3,31,41]' should equal '[3,41,31]'
|-
|-
| {{bug|838192}} [[https://github.com/mozilla/mozmill-dashboard/pull/50 Pull Request]] || Review || Henrik/Dave || Andrei || Add screenshot functionality to assertion module
| {{bug|838192}} [[https://github.com/mozilla/mozmill-dashboard/pull/50 Pull Request]] || Review || Henrik/Dave || Andrei || Add screenshot functionality to assertion module
|-
|-
| {{bug|599290}} || Feedback || Henrik || Cosmin || Update SoftwareUpdateAPI to support complete relocation to the about dialog
| {{bug|599290}} || Feedback || Henrik || Cosmin || Update SoftwareUpdateAPI to support complete relocation to the about dialog
|-
| {{bug|874393}} || Feedback || Henrik/Dave || Mario || [Australis] Update SoftwareUpdateAPI to support complete relocation to the about dialog
|}
|}


== Mobile (Robocop) ==
== Mobile (Robocop) ==
* Goals progress:
* Progress on [https://docs.google.com/spreadsheet/ccc?key=0AhE7m4JB2j6tdDJBT2dlbVJwUk9PSy1RbHo4WVNiUGc#gid=8 Robocop Automation]
* TBD
* Bugs with work in progress:
** {{Bug|820859}} - Robocop: Add test for 'Clear Private Data'
** {{Bug|830834}} - Robocop: Add test for 'Show Search Suggestions' feature
** {{Bug|836341}} - Robocop: Add test for "Clear download history"
** {{Bug|869277}} - Intermittent testAddSearchEngine | The number of search results has increased - got 5, expected 10
** {{Bug|854043}} - Intermittent testWebContentContextMenu | Verify that the download started - The download started
* Bugs awaiting integration:
** {{Bug|888277}} - Robocop: Add test for Private Browsing
* Bugs recently integrated:
** {{Bug|886299}} - Intermittent testShareLink | Exception caught - junit.framework.AssertionFailedError: The text: Share is not found!
* '''Issues'''
** Because of {{Bug|891889}} - ''Only running 2 of 3 chunks for Android 4.0 robocop'' - we can't integrate any new tests since they are not run on the tryserver
** I was unable to build the armv6 version in order to debug {{Bug|869277}} - the testAddSearchEngine intermittent fail - logged a new issue: {{Bug|893759}} - Unable to build armv6 on linux x86_64 from the latest m-c sources
** The implementation of the new multipage about:home on mobile will most likely cause a need to rewrite most of the tests


= Personal Status =
= Personal Status =
Line 121: Line 152:


==Action items==
==Action items==
* {{new|}} %name%: %task%
* {{prev|}} Henrik - Start a thread on the public list to collect ideas for the show and tell
* {{prev|}} Dave, Rob - Discuss gaia-ui tests, autolog, heuristics and html reports
* {{new|}} Henrik - talk to Clint about next quarter goals
* {{new|}} Andreea - start a mail thread about the bug priorities

Latest revision as of 16:32, 15 July 2013

previous meeting | Meetings | next meeting »

Dial in

 # When:     Every Monday at 8:45am PDT/PST
 # Vidyo:    https://v.mozilla.com/flex.html?roomdirect.html&key=PGtLpx3XQGJz
 # Phone:    650-903-0800 or 650-215-1282 x92 Conf# 9654 (US/INTL)
 #           1-800-707-2533 (pin 369) Conf# 9654 (US)
 # IRC:      irc://irc.mozilla.org:6697/#automation
 # IRC logs: http://irclog.gr/#browse/irc.mozilla.org/automation

General

  • Attendees:
    • Dave, Andrei, Daniela, Andreea, Rob, Adrian, Henrik, Cosmin, Otilia
  • Chair: Henrik
  • Notes: Andreea

Last Weeks Action Items

  • [CARRY OVER] Henrik - Start a thread on the public list to collect ideas for the show and tell
  • [DONE] Henrik - Develop patch/update to only enable Mozmill restarts and not user-initiated restarts
    • Not sure what this entry was about, but I'm not going to fix that for 2.0. The bug has been filed btw.
  • [CARRY OVER] Dave, Rob - Discuss gaia-ui tests, autolog, heuristics and html reports

Highlights

  • [henrik] Due to the constant failures in submitting Mozmill test results to iriscouch we have been moved to a new server. Big thanks goes to Jason Smith from IrisCouch for his ultra-fast response and action!
  • [henrik] About to fix a major bug in jsbridge which leds to dataloss when sending too much data. See bug 764640.
  • [dave] First passing Travis-CI build for Gaia UI tests on desktop build!
  • [dave] HTML report for Gaia UI tests finally available!
  • [dave] Gaia UI tests are now running (and passing!) in TBPL against the Cedar branch
  • [dave] b2gpopulate 0.8 released with support for call history and MMS messages
  • [dave] b2gperf 0.5 released
  • New blockers, any updates?
    • bug 892928 - b2gperf is failing to gather FPS measurements on mozilla-central builds
    • bug 893693 - TypeError: IccHelper.removeEventListener is not a function
  • You can find all blockers here: http://bit.ly/Tph0NY

Roundtable

  • [Andreea] Bug Priorities
    • P1 - critical, constant failures, important enhancements (regressions, tests that fail constantly. I'd say even if skipped)
    • P2 - mozmill 2.0 blocker (if release is approaching, this can take higher priority in order to finish in time)
    • P3 - not that often failures, medium enhancements
    • P4 - new tests - the ones needed by the QA team have higher priority
    • P5 - refactoring, small enhancements
    • Dependecies take the blocking bug's priority
  • [henrik] Do we want to make use of github releases instead of uploading versions to pypi directly?

Goals Overview

Project Updates

Gaia UI (Dave)

  • Goals progress:
  • See highlights

WebAPI (Rob)

  • Submitted patch to fix two WebTelephony WebAPI failures

WebRTC (Henrik)

  • In process to get the datachannel framework backported - currently blocked by try outage
  • Most likely working on two or three other enhancements

Games Benchmarking (Rob, Joel, BC)

  • [rwood] Working on getting speedtests running on Firefox OS

Softvision Updates

Desktop Automation

  • Goals progress: Mozmill 2.0 releases
  • Fixed:
    • bug 882068 - Test failure "linkId.getNode(...) is null" in testSessionStore/testUndoTabFromContextMenu.js with Mozmill 2.0
    • bug 887315 - Mutt test window_focus.js fails with: "Windows have maintained their order - '[3,31,41]' should equal '[3,41,31]'
    • bug 887258 - Disable new tab preloading preference in mozmill 1.5
    • bug 890883 - Test failure due to timeout when trying to find the Find Bar in testFindInPage
    • bug 889314 - Update the restartless.xpi addon to be compatible with AMO. Set maxValue to a valid value instead of "*"
    • bug 891842 - Test failure 'Expression "{"anonid":"findbar-textbox"}" returned null. Anonymous == true' in testFindInPage.js
    • bug 892946 - Test failure in testFindInPage due to the fact that the find bar was moved to the top
    • bug 892388 - Create a clone of restartless.xpi for uploading to AMO with an end user license agreement
    • bug 763461 - Mozmill tests for cut-off elements should not report failure for scrollable direction
  • Working on:
    • Daniela/Cosmin
      • bug 885221 - controller.waitForPageLoad() failures with mozmill 2.0 due to closeAllTabs method opening about:newtab instead of about:blank
      • bug 880135 - Test failure "The tab with index '1' has been selected" in testOpenInBackground.js
      • bug 684801 - Timeout failure in /testTabView_SwitchTabs/test1.js | TabView is still open.
    • Andreea
      • bug 795398 - Mozmill test to check that MD5 hash signatures are no longer accepted
      • bug 880417 - Create user interface shared module for metro
      • bug 853005 - Update automation script should exit with a non-zero exit code if unable to get update channel
    • Mario
      • bug 874394 - [Australis] Test failure in '/testAwesomeBar/testPasteLocationBar.js' due to the urlbar-container not being found
      • bug 835296 - Add test that installs an addon with EULA directly from addons.mozilla.org
      • bug 835383 - Add test that installs an addon without EULA directly from addons.mozilla.org
      • bug 835132 - Test failure "Current domain name matches target domain name" in /testSearch/testSearchSelection.js
    • Andrei
      • bug 888196 - Memory increase in endurance testsruns on Win 8 (since June the 16th) - running more endurance tests on SCL
Waiting for review/Feedback/Need info
Bug number Status Requestee Requester Bug Subject
bug 489474 Review Henrik Andreea Write automated tests for user generated video controls
bug 800872 Review Henrik Andreea Test failure "Window number '1' has been opened" in endurance/testTabbedBrowsing_OpenNewWindow/test1.js
bug 786306 Review Henrik Mario Restart tests logic
bug 732353 Feedback Henrik Daniela Disable all Discovery Pane tests due to unpredictable web dependencies
bug 799149 Review Henrik Daniela Add mozmill test to ensure that there are no conflicts in context menu access keys.
bug 791634 Review Henrik Daniela Make waitFor() method available in the assertions module (also for soft assertions)
bug 871441 Feedback Henrik Daniela Focus inside the select method is not set properly on the menulist
bug 886811 Review Henrik Andrei Use controller.restartAppplication() instead of controller.startUserShutdown() for mozmill-2.0
bug 838192 [Pull Request] Review Henrik/Dave Andrei Add screenshot functionality to assertion module
bug 599290 Feedback Henrik Cosmin Update SoftwareUpdateAPI to support complete relocation to the about dialog
bug 874393 Feedback Henrik/Dave Mario [Australis] Update SoftwareUpdateAPI to support complete relocation to the about dialog

Mobile (Robocop)

  • Progress on Robocop Automation
  • Bugs with work in progress:
    • bug 820859 - Robocop: Add test for 'Clear Private Data'
    • bug 830834 - Robocop: Add test for 'Show Search Suggestions' feature
    • bug 836341 - Robocop: Add test for "Clear download history"
    • bug 869277 - Intermittent testAddSearchEngine | The number of search results has increased - got 5, expected 10
    • bug 854043 - Intermittent testWebContentContextMenu | Verify that the download started - The download started
  • Bugs awaiting integration:
    • bug 888277 - Robocop: Add test for Private Browsing
  • Bugs recently integrated:
    • bug 886299 - Intermittent testShareLink | Exception caught - junit.framework.AssertionFailedError: The text: Share is not found!
  • Issues
    • Because of bug 891889 - Only running 2 of 3 chunks for Android 4.0 robocop - we can't integrate any new tests since they are not run on the tryserver
    • I was unable to build the armv6 version in order to debug bug 869277 - the testAddSearchEngine intermittent fail - logged a new issue: bug 893759 - Unable to build armv6 on linux x86_64 from the latest m-c sources
    • The implementation of the new multipage about:home on mobile will most likely cause a need to rewrite most of the tests

Personal Status

For the personal status please check the weekly status updates:

Meeting Notes

The meeting notes for this meeting can be read in the following etherpad:

Action items

  • [CARRY OVER] Henrik - Start a thread on the public list to collect ideas for the show and tell
  • [CARRY OVER] Dave, Rob - Discuss gaia-ui tests, autolog, heuristics and html reports
  • [NEW] Henrik - talk to Clint about next quarter goals
  • [NEW] Andreea - start a mail thread about the bug priorities
Retrieved from "https://wiki.allizom.org/index.php?title=Auto-tools/Automation_Development/Meetings/130715&oldid=676308"