Auto-tools/Automation Development/Meetings/130415: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(Created page with "<small>previous meeting | Meetings | [[Auto-tools/Automation_Development/M...")
 
 
(25 intermediate revisions by 7 users not shown)
Line 2: Line 2:


= Dial in =
= Dial in =
  # Vidyo: https://v.mozilla.com/flex.html?roomdirect.html&key=PGtLpx3XQGJz
{{:Auto-tools/Automation_Development/Meeting_Details}}
  # Phone:  650-903-0800 or 650-215-1282 x92 Conf# 9654 (US/INTL)
  #        1-800-707-2533 (pin 369) Conf# 9654 (US)
  # IRC:    irc://irc.mozilla.org:6697/#automation


= General =
= General =
* Attendees:  
* Attendees:  
** TBD
** TBD
* Chair: Dave Hunt
* Chair: Dave
* Notes: Andreea Matei
* Notes: Andreea


= Last Weeks Action Items =
= Last Weeks Action Items =
Line 17: Line 14:


= Highlights =
= Highlights =
* [%name%] %highlight%
* [henrik] As of Thursday last week pulsebuildmonitor support for CI is now live on production!
* New blockers, any updates?
* New blockers, any updates?
** %new blockers%
**  
* You can find all blockers here: http://bit.ly/Tph0NY
* You can find all blockers here: http://bit.ly/Tph0NY


= Roundtable =
= Roundtable =
* Mozmil 2.0? triage
* [https://bugzilla.mozilla.org/buglist.cgi?resolution=---&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=mozmill-2.0%2B&component=Mozmill&product=Testing&list_id=6311401 Mozmill 2.0+ traige]
* [https://bugzilla.mozilla.org/buglist.cgi?resolution=---;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-2.0%3F;product=Testing;list_id=3040650 Mozmill 2.0? triage]
* Mozmill on Metro


= Goals Overview =
= Goals Overview =
Line 32: Line 31:
== Gaia UI (Dave) ==
== Gaia UI (Dave) ==
* Goals progress:
* Goals progress:
* TBD
* [dave] Submitted a patch to remove indexedDB files between tests
* [dave] Submitted a patch to clarify online/offline requirements
* [dave] Malini is helping out with some desktop related fixes
* [rwood] Gaia-ui long-running tests:
** Developed a test for switching from gallery to camera and back, as requested by dev ({{bug|851626}})
** Improving test stability
** Will be submitting first set of results to datazilla in the next day or two
* [rwood] Developing gaia-ui tests for Bluetooth discovery and pairing using host-side bluetooth controller


== WebAPI (Rob) ==
== WebAPI (Rob) ==
* Goals progress:
* [rwood] Fixed test_proximity_change intermittent failure ({{bug|851658}})
* TBD
* [rwood] Increased all WebSms and WebTelephony test timeouts in order to reduce intermittent test failures ({{bug|841285}} and {{bug|844490}})


== WebRTC (Henrik) ==
== WebRTC (Henrik) ==
* Goals progress:
* Found spec issue with data channels
* TBD
* Will focus on datachannel this week (given that I'm not duty this week)


= Softvision Updates =
= Softvision Updates =
== Desktop Automation ==
== Desktop Automation ==
* Goals progress:
* Goals progress: Q1 - achieved,  Q2 - Mozmill 2.0 release
* TBD
 
* Fixed:
** {{bug|855325}} - Missing nspr4 module causes jsbridge to fail when creating the network socket
** {{bug|826251}} - Make testPreferences/testPreferredLanguage.js language independend
 
* Blocked bugs:
** {{bug|763159}} - Test failure 'secure.mur.at == erle.mur.at ' in testSecurity/testUnknownIssuer.js
** {{bug|795398}} - Mozmill test to check that MD5 hash signatures are no longer accepted
*** needs CA certificate
 
* Working on:
** [https://github.com/mozilla/mozmill-ci/issues/229 Mozmill-ci] - Log request/response bodies of update requests
*** trying with EnvInject plugin
** {{bug|840022}} - "The forward button has been made visible for the 1 page"
** {{bug|859241}} - Update addons.js to enhance the way addonsManager.close() works
** {{bug|786306}} - Add more logic to the restart tests to skip following test files if a test is failing
** {{bug|599290}} - Update SoftwareUpdateAPI to support complete relocation to the about dialog
** {{bug|842571}} - Update the process start directory (XCurProcD) in software-update.js library
** {{bug|732353}} - Disable all Discovery Pane tests due to unpredictable web dependencies
** {{bug|857551}} - Create HTML page containing all the elements necessary for L10N testing of the context menu access keys
** {{bug|599290}} - Update SoftwareUpdateAPI to support complete relocation to the about dialog
** {{bug|848577}} - Create a Mozmill Endurance test to catch regular expression memory leak
** {{bug|763470}} - False positive test for cut-off elements for advanced pane tabs
 
{| border="1" cellpadding="2" align="center"
|+<b> Waiting for review/Feedback/Need info</b>
| <b>Bug number</b> || <b>Status</b>  || <b>People</b>  || <b>Bug Subject</b>
|-
| {{bug|489474}} || Review || Henrik || Write automated tests for user generated video controls
|-
| {{bug|800872}} || Review || Henrik || Test failure "Window number '1' has been opened" in endurance/testTabbedBrowsing_OpenNewWindow/test1.js
|-
| {{bug|838192}} [[https://github.com/mozilla/mozmill-dashboard/pull/50 Pull Request]] || Review || Henrik/Dave || Add screenshot functionality to assertion module
|-
| {{bug|839996}} || Feedback || Marco || Add event to wait for before importing the default bookmarks through importFromURL function
|-
| {{bug|838192}} || Review || Henrik/Dave || Add screenshot functionality to assertion module (low priority with mozmill 2.0 looming)
|-
| [https://github.com/whimboo/mozmill-automation/pull/32 Pull request] || Review || Henrik/Dave || Update automation script should exit with a non-zero exit code if unable to get update channel (Mozmill 2.0)
|-
| {{bug|834163}} || Need-info || Henrik || Teardown code for /testSearch tests should not exercise ui path which can cause failures like: "Modal dialog has been found and processed"
|-
| {{bug|785014}} || Need-info || Alexandre Poirot || Build restartless.xpi as a plain bootstrapped extension
|-
| {{bug|853478}} || Review || Henrik/Dave || Junit report generation fails if Mozmill is not able to send the report to the dashboard
|-
| {{bug|795398}} || Review || Henrik/Dave || Mozmill test to check that MD5 hash signatures are no longer accepted - not for landing yet
|-
| {{bug|859746}} || Review || Henrik/Dave || Mutt javascript tests do not print the RESULTS block
|-
| {{bug|803492}} || Needinfo || Neil Deakin || Enhance the constructor of MozMillController to set focus on the window
|}
 
== Mobile (Robocop) ==
== Mobile (Robocop) ==
* Goals progress:
* Progress on [https://docs.google.com/spreadsheet/ccc?key=0AhE7m4JB2j6tdDJBT2dlbVJwUk9PSy1RbHo4WVNiUGc#gid=8 Robocop Automation]
* TBD
* Bugs with work in progress:
** {{Bug|820859}} - Robocop: Add test for 'Clear Private Data'
** {{Bug|830834}} - Robocop: Add test for 'Show Search Suggestions' feature
** {{Bug|846340}} - Robocop: Add test for 'Clear Site Settings'
** {{Bug|836341}} - Robocop: Add test for "Clear download history"
** {{Bug|838596}} - Robocop: Add test for 'Master Password' feature
** {{Bug|846296}} - Robocop: Add test for 'Add Search Engine' feature
** {{Bug|833832}} - Robocop: Add test for Import from Android feature
** {{Bug|855978}} - Robocop: upgrade to robotium-solo-4.0.jar
** {{Bug|836818}} - Intermittent testOverscroll | Checking gecko didn't draw unnecessarily - got true, expected false
** {{Bug|851861}} - Intermittent testOverscroll, testPanCorrectness, testAxisLocking, testLoad, testFlingCorrectness | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
*Notes/Issues/Questions:
** We looked into a patch to add robotium 4.1 to Robocop but we are still having problems in the try-runs
** We managed to fix the build issue ({{Bug|859341}}) - There was a problem building on 64 bit systems using the 32 bit toolchain. We upgraded to NDK r8e which contains a 64 bit toolchain. We also did some investigations for the related bugs.
** We still don't have an update on the robocop build issue - {{Bug|855306}} - Robocop build does not re-build tests - still no update from :joey - we are still working with older revisions or rebuilding everything
** blockForGeckoReady seems to unblock to early - there are issues when loading larger web pages (like Robocop Boxes). The simple solution is to load a blank page first.
** in testPanCorrectness and testAxisLocking I am notices on devices that the drag over 100px moves only 90+ px although on the tryserver the tests seem to pass
** we are still seeing issues with the "Private data cleared" toaster notification not being displayed - {{Bug|837274}}. The "Clear private data" tests on tryserver seem to constantly fail.


= Personal Status =
= Personal Status =
Line 59: Line 134:


==Action items==
==Action items==
* {{new|}} %name%: %task%
* {{new|}} Henrik - Update the goals page
* {{prev|}} Anthony: Ensure any Metro tests are suitably tagged with in-testsuite?

Latest revision as of 15:11, 18 April 2013

previous meeting | Meetings | next meeting »

Dial in

 # When:     Every Monday at 8:45am PDT/PST
 # Vidyo:    https://v.mozilla.com/flex.html?roomdirect.html&key=PGtLpx3XQGJz
 # Phone:    650-903-0800 or 650-215-1282 x92 Conf# 9654 (US/INTL)
 #           1-800-707-2533 (pin 369) Conf# 9654 (US)
 # IRC:      irc://irc.mozilla.org:6697/#automation
 # IRC logs: http://irclog.gr/#browse/irc.mozilla.org/automation

General

  • Attendees:
    • TBD
  • Chair: Dave
  • Notes: Andreea

Last Weeks Action Items

  • [CARRY OVER] Anthony: Ensure any Metro tests are suitably tagged with in-testsuite?

Highlights

  • [henrik] As of Thursday last week pulsebuildmonitor support for CI is now live on production!
  • New blockers, any updates?
  • You can find all blockers here: http://bit.ly/Tph0NY

Roundtable

Goals Overview

Project Updates

Gaia UI (Dave)

  • Goals progress:
  • [dave] Submitted a patch to remove indexedDB files between tests
  • [dave] Submitted a patch to clarify online/offline requirements
  • [dave] Malini is helping out with some desktop related fixes
  • [rwood] Gaia-ui long-running tests:
    • Developed a test for switching from gallery to camera and back, as requested by dev (bug 851626)
    • Improving test stability
    • Will be submitting first set of results to datazilla in the next day or two
  • [rwood] Developing gaia-ui tests for Bluetooth discovery and pairing using host-side bluetooth controller

WebAPI (Rob)

  • [rwood] Fixed test_proximity_change intermittent failure (bug 851658)
  • [rwood] Increased all WebSms and WebTelephony test timeouts in order to reduce intermittent test failures (bug 841285 and bug 844490)

WebRTC (Henrik)

  • Found spec issue with data channels
  • Will focus on datachannel this week (given that I'm not duty this week)

Softvision Updates

Desktop Automation

  • Goals progress: Q1 - achieved, Q2 - Mozmill 2.0 release
  • Fixed:
    • bug 855325 - Missing nspr4 module causes jsbridge to fail when creating the network socket
    • bug 826251 - Make testPreferences/testPreferredLanguage.js language independend
  • Blocked bugs:
    • bug 763159 - Test failure 'secure.mur.at == erle.mur.at ' in testSecurity/testUnknownIssuer.js
    • bug 795398 - Mozmill test to check that MD5 hash signatures are no longer accepted
      • needs CA certificate
  • Working on:
    • Mozmill-ci - Log request/response bodies of update requests
      • trying with EnvInject plugin
    • bug 840022 - "The forward button has been made visible for the 1 page"
    • bug 859241 - Update addons.js to enhance the way addonsManager.close() works
    • bug 786306 - Add more logic to the restart tests to skip following test files if a test is failing
    • bug 599290 - Update SoftwareUpdateAPI to support complete relocation to the about dialog
    • bug 842571 - Update the process start directory (XCurProcD) in software-update.js library
    • bug 732353 - Disable all Discovery Pane tests due to unpredictable web dependencies
    • bug 857551 - Create HTML page containing all the elements necessary for L10N testing of the context menu access keys
    • bug 599290 - Update SoftwareUpdateAPI to support complete relocation to the about dialog
    • bug 848577 - Create a Mozmill Endurance test to catch regular expression memory leak
    • bug 763470 - False positive test for cut-off elements for advanced pane tabs
Waiting for review/Feedback/Need info
Bug number Status People Bug Subject
bug 489474 Review Henrik Write automated tests for user generated video controls
bug 800872 Review Henrik Test failure "Window number '1' has been opened" in endurance/testTabbedBrowsing_OpenNewWindow/test1.js
bug 838192 [Pull Request] Review Henrik/Dave Add screenshot functionality to assertion module
bug 839996 Feedback Marco Add event to wait for before importing the default bookmarks through importFromURL function
bug 838192 Review Henrik/Dave Add screenshot functionality to assertion module (low priority with mozmill 2.0 looming)
Pull request Review Henrik/Dave Update automation script should exit with a non-zero exit code if unable to get update channel (Mozmill 2.0)
bug 834163 Need-info Henrik Teardown code for /testSearch tests should not exercise ui path which can cause failures like: "Modal dialog has been found and processed"
bug 785014 Need-info Alexandre Poirot Build restartless.xpi as a plain bootstrapped extension
bug 853478 Review Henrik/Dave Junit report generation fails if Mozmill is not able to send the report to the dashboard
bug 795398 Review Henrik/Dave Mozmill test to check that MD5 hash signatures are no longer accepted - not for landing yet
bug 859746 Review Henrik/Dave Mutt javascript tests do not print the RESULTS block
bug 803492 Needinfo Neil Deakin Enhance the constructor of MozMillController to set focus on the window

Mobile (Robocop)

  • Progress on Robocop Automation
  • Bugs with work in progress:
    • bug 820859 - Robocop: Add test for 'Clear Private Data'
    • bug 830834 - Robocop: Add test for 'Show Search Suggestions' feature
    • bug 846340 - Robocop: Add test for 'Clear Site Settings'
    • bug 836341 - Robocop: Add test for "Clear download history"
    • bug 838596 - Robocop: Add test for 'Master Password' feature
    • bug 846296 - Robocop: Add test for 'Add Search Engine' feature
    • bug 833832 - Robocop: Add test for Import from Android feature
    • bug 855978 - Robocop: upgrade to robotium-solo-4.0.jar
    • bug 836818 - Intermittent testOverscroll | Checking gecko didn't draw unnecessarily - got true, expected false
    • bug 851861 - Intermittent testOverscroll, testPanCorrectness, testAxisLocking, testLoad, testFlingCorrectness | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
  • Notes/Issues/Questions:
    • We looked into a patch to add robotium 4.1 to Robocop but we are still having problems in the try-runs
    • We managed to fix the build issue (bug 859341) - There was a problem building on 64 bit systems using the 32 bit toolchain. We upgraded to NDK r8e which contains a 64 bit toolchain. We also did some investigations for the related bugs.
    • We still don't have an update on the robocop build issue - bug 855306 - Robocop build does not re-build tests - still no update from :joey - we are still working with older revisions or rebuilding everything
    • blockForGeckoReady seems to unblock to early - there are issues when loading larger web pages (like Robocop Boxes). The simple solution is to load a blank page first.
    • in testPanCorrectness and testAxisLocking I am notices on devices that the drag over 100px moves only 90+ px although on the tryserver the tests seem to pass
    • we are still seeing issues with the "Private data cleared" toaster notification not being displayed - bug 837274. The "Clear private data" tests on tryserver seem to constantly fail.

Personal Status

For the personal status please check the weekly status updates:

Meeting Notes

The meeting notes for this meeting can be read in the following etherpad:

Action items

  • [NEW] Henrik - Update the goals page
  • [CARRY OVER] Anthony: Ensure any Metro tests are suitably tagged with in-testsuite?
Retrieved from "https://wiki.allizom.org/index.php?title=Auto-tools/Automation_Development/Meetings/130415&oldid=647756"