Auto-tools/Automation Development/Meetings/130422: Difference between revisions

Jump to navigation Jump to search
 
(11 intermediate revisions by 4 users not shown)
Line 18: Line 18:
* New blockers, any updates?
* New blockers, any updates?
** {{bug|862724}} - Additional pulse message for same nightly builds sent (with previous_buildid == buildid)
** {{bug|862724}} - Additional pulse message for same nightly builds sent (with previous_buildid == buildid)
** {{bug|864294}} - Activities database is not automatically recreated if it has been removed
* You can find all blockers here: http://bit.ly/Tph0NY
* You can find all blockers here: http://bit.ly/Tph0NY


Line 31: Line 32:
*** Support for (remote) debugging
*** Support for (remote) debugging
*** Changes of getting a much broader community around desktop testing
*** Changes of getting a much broader community around desktop testing
*** Marionette has more active contributors than Mozmill
** contra
** contra
*** First we have to figure out how to inject Marionette in default Firefox builds (see bug 836253)
*** First we have to figure out how to inject Marionette in default Firefox builds (see {{bug|836253}})
*** Existing Mozmill tests would have to be transformed or we keep and run them in parallel (some tests would even be not necessary anymore due to new browser chrome tests)
*** Existing Mozmill tests would have to be transformed or we keep and run them in parallel (some tests would even be not necessary anymore due to new browser chrome tests)
*** Where would we publish our test results? Our dashboard would need a rewrite or we can directly push to tbpl
*** Where would we publish our test results? Our dashboard would need a rewrite or we can directly push to tbpl
**** Probably TBPL2 (Tree Herder)
* Mozmill 2.0 triage part 2


= Goals Overview =
= Goals Overview =
Line 44: Line 48:
* Goals progress:
* Goals progress:
* TBD
* TBD
* [dave] Marionette client 0.5.24 released with several fixes for running against B2G desktop builds
* [dave] Database cleanup pull is currently blocked by {{bug|864294}}
* [rwood] Submitted pull request for gaia-ui Bluetooth device discoverable test
* [rwood] Submitted pull request for gaia-ui Bluetooth device discoverable test
* [rwood] Gaia-ui endurance tests:
* [rwood] Gaia-ui endurance tests:
** First results submissions to DataZilla b2g project
** First results submissions to DataZilla b2g project
** Setting up jenkins on my test PC
** Setting up jenkins on my test PC
** Tests to be submitted for review soon
** Tests submitted for review
** Once the tests are in the gaia-ui repo and more results are on DataZilla, I will give a talk about them in our meeting, write a blog post, etc.
** Once the tests are in the gaia-ui repo and more results are on DataZilla, I will give a talk about them in our meeting, write a blog post, etc.


Line 69: Line 75:
** {{bug|708270}} - Flash endurance tests expose potential performance degradation regression
** {{bug|708270}} - Flash endurance tests expose potential performance degradation regression
** {{bug|803489}} - Software update tests on Windows 8 fail sometimes due to updater prompt on startup (jsbridge cannot connect)
** {{bug|803489}} - Software update tests on Windows 8 fail sometimes due to updater prompt on startup (jsbridge cannot connect)
** {{bug|826251}} - Make testPreferences/testPreferredLanguage.js language independend
** {{bug|859746}} - Mutt javascript tests do not print the RESULTS block


* Blocked bugs:
* Blocked bugs:
Line 78: Line 86:
** [https://github.com/mozilla/mozmill-ci/issues/229 Mozmill-ci] - Log request/response bodies of update requests
** [https://github.com/mozilla/mozmill-ci/issues/229 Mozmill-ci] - Log request/response bodies of update requests
*** trying with EnvInject plugin  
*** trying with EnvInject plugin  
** {{bug|758187}} - Write a new mozmill test for checking location sharing
** {{bug|860664}} - controller.waitThenClick() does not correctly wait for the element
** {{bug|860655}} - elementslib.ID() fails to grab the element - "could not find element ID: id"
** {{bug|732353}} - Disable all Discovery Pane tests due to unpredictable web dependencies
** {{bug|732353}} - Disable all Discovery Pane tests due to unpredictable web dependencies
** {{bug|857551}} - Create HTML page containing all the elements necessary for L10N testing of the context menu access keys
** {{bug|857551}} - Create HTML page containing all the elements necessary for L10N testing of the context menu access keys
Line 86: Line 95:
** {{bug|860659}} - elementslib.Lookup() fails with 'Expression \"id(\"main-window\")\" returned null. Anonymous == false"'
** {{bug|860659}} - elementslib.Lookup() fails with 'Expression \"id(\"main-window\")\" returned null. Anonymous == false"'
** {{bug|791634}} - Make waitFor() method available in the assertions module (also for soft assertions)
** {{bug|791634}} - Make waitFor() method available in the assertions module (also for soft assertions)
** {{bug|763461}} - Mozmill tests for cut-off elements should not report failure for scrollable direction
** {{bug|763470}} - False positive test for cut-off elements for advanced pane tabs
** {{bug|764640}} - Mozmill returns PASS/SKIP/FAIL == 0 for test_assertions.js
** {{bug|860670}} - mozmill.utils.getWindowByType fails for type: Addons:Blocklist - Fails with "Window has been found."
** {{bug|860662}} - controller.select() fails to select the option by value


{| border="1" cellpadding="2" align="center"
{| border="1" cellpadding="2" align="center"
Line 100: Line 114:
|-
|-
| {{bug|838192}} || Review || Henrik/Dave || Add screenshot functionality to assertion module (low priority with mozmill 2.0 looming)
| {{bug|838192}} || Review || Henrik/Dave || Add screenshot functionality to assertion module (low priority with mozmill 2.0 looming)
|-
| [https://github.com/whimboo/mozmill-automation/pull/32 Pull request] || Review || Henrik/Dave || Update automation script should exit with a non-zero exit code if unable to get update channel (Mozmill 2.0)
|-
| {{bug|834163}} || Need-info || Henrik || Teardown code for /testSearch tests should not exercise ui path which can cause failures like: "Modal dialog has been found and processed"
|-
|-
| {{bug|785014}} || Need-info || Alexandre Poirot || Build restartless.xpi as a plain bootstrapped extension
| {{bug|785014}} || Need-info || Alexandre Poirot || Build restartless.xpi as a plain bootstrapped extension
Line 116: Line 126:
|-
|-
| {{bug|860665}} || Feedback || Henrik/Dave || controller.waitForPageLoad() fails when working with Private Browsing windows.
| {{bug|860665}} || Feedback || Henrik/Dave || controller.waitForPageLoad() fails when working with Private Browsing windows.
|-
| {{bug|827276}} || Needinfo || Alex Keybl || Test failure 'Check for updates has been completed' in /restartTests/testSoftwareUpdateAutoProxy/test2.js
|}
|}


Line 151: Line 163:


==Action items==
==Action items==
* {{new|}} %name%: %task%
* {{prev|}} Henrik - Update the goals page
* {{prev|}} Anthony: Ensure any Metro tests are suitably tagged with in-testsuite?
* {{new|}} Andreea: File bug about the broken elementslib handling with elements
canmove, Confirmed users, Bureaucrats and Sysops emeriti
4,714

edits

Navigation menu