canmove, Confirmed users, Bureaucrats and Sysops emeriti
4,714
edits
Auto-tools/Automation Development/Meetings/130422: Difference between revisions
Auto-tools/Automation Development/Meetings/130422 (view source)
Revision as of 09:23, 23 April 2013
, 23 April 2013→Action items
| (18 intermediate revisions by 5 users not shown) | |||
| Line 15: | Line 15: | ||
= Highlights = | = Highlights = | ||
* [ | * [dave] Marionette client 0.5.24 released with several fixes for running tests on B2G desktop builds | ||
* New blockers, any updates? | * New blockers, any updates? | ||
** {{bug|862724}} - Additional pulse message for same nightly builds sent (with previous_buildid == buildid) | ** {{bug|862724}} - Additional pulse message for same nightly builds sent (with previous_buildid == buildid) | ||
** {{bug|864294}} - Activities database is not automatically recreated if it has been removed | |||
* You can find all blockers here: http://bit.ly/Tph0NY | * You can find all blockers here: http://bit.ly/Tph0NY | ||
= Roundtable = | = Roundtable = | ||
* | * How to implement Metro tests, as it looks like we will majorly be blocked on missing event actions in Mozmill. Should we consider Marionette instead? | ||
* [henrik] Really awful proposal - Shall we drop work on Mozmill and start with Marionette for tests? | |||
** pro arguments | |||
*** Only a few tests are using complex mouse events like drag&drop, and we do not support touch events at all in Mozmill | |||
*** Marionette gets injected in the system process as extra library and doesn't have to be build as extension, which even is not supported for all products (e.g native web apps don't allow add-ons to be installed) | |||
*** W3C compliance by using the web driver protocol (http://www.w3.org/TR/2013/WD-webdriver-20130117/) | |||
*** Tests can even be run on mobile (Android, B2G) | |||
*** Tests can be written in different languages (e.g. Python) and allow us to write system integration tests because we are not in the browser scope anymore | |||
*** Support for (remote) debugging | |||
*** Changes of getting a much broader community around desktop testing | |||
*** Marionette has more active contributors than Mozmill | |||
** contra | |||
*** First we have to figure out how to inject Marionette in default Firefox builds (see {{bug|836253}}) | |||
*** Existing Mozmill tests would have to be transformed or we keep and run them in parallel (some tests would even be not necessary anymore due to new browser chrome tests) | |||
*** Where would we publish our test results? Our dashboard would need a rewrite or we can directly push to tbpl | |||
**** Probably TBPL2 (Tree Herder) | |||
* Mozmill 2.0 triage part 2 | |||
= Goals Overview = | = Goals Overview = | ||
| Line 31: | Line 48: | ||
* Goals progress: | * Goals progress: | ||
* TBD | * TBD | ||
* [dave] Marionette client 0.5.24 released with several fixes for running against B2G desktop builds | |||
* [dave] Database cleanup pull is currently blocked by {{bug|864294}} | |||
* [rwood] Submitted pull request for gaia-ui Bluetooth device discoverable test | |||
* [rwood] Gaia-ui endurance tests: | |||
** First results submissions to DataZilla b2g project | |||
** Setting up jenkins on my test PC | |||
** Tests submitted for review | |||
** Once the tests are in the gaia-ui repo and more results are on DataZilla, I will give a talk about them in our meeting, write a blog post, etc. | |||
== WebAPI (Rob) == | == WebAPI (Rob) == | ||
* | * No updates this week | ||
== WebRTC (Henrik) == | == WebRTC (Henrik) == | ||
* | * Caught a major regression ({{bug|863687}}) and leak for data channels last week (a single char change) | ||
* | * Framework 70% done, but needs further cleanup before the first review | ||
= Softvision Updates = | = Softvision Updates = | ||
| Line 51: | Line 75: | ||
** {{bug|708270}} - Flash endurance tests expose potential performance degradation regression | ** {{bug|708270}} - Flash endurance tests expose potential performance degradation regression | ||
** {{bug|803489}} - Software update tests on Windows 8 fail sometimes due to updater prompt on startup (jsbridge cannot connect) | ** {{bug|803489}} - Software update tests on Windows 8 fail sometimes due to updater prompt on startup (jsbridge cannot connect) | ||
** {{bug|826251}} - Make testPreferences/testPreferredLanguage.js language independend | |||
** {{bug|859746}} - Mutt javascript tests do not print the RESULTS block | |||
* Blocked bugs: | * Blocked bugs: | ||
| Line 60: | Line 86: | ||
** [https://github.com/mozilla/mozmill-ci/issues/229 Mozmill-ci] - Log request/response bodies of update requests | ** [https://github.com/mozilla/mozmill-ci/issues/229 Mozmill-ci] - Log request/response bodies of update requests | ||
*** trying with EnvInject plugin | *** trying with EnvInject plugin | ||
** {{bug| | ** {{bug|860664}} - controller.waitThenClick() does not correctly wait for the element | ||
** {{bug|860655}} - elementslib.ID() fails to grab the element - "could not find element ID: id" | |||
** {{bug|732353}} - Disable all Discovery Pane tests due to unpredictable web dependencies | ** {{bug|732353}} - Disable all Discovery Pane tests due to unpredictable web dependencies | ||
** {{bug|857551}} - Create HTML page containing all the elements necessary for L10N testing of the context menu access keys | ** {{bug|857551}} - Create HTML page containing all the elements necessary for L10N testing of the context menu access keys | ||
| Line 68: | Line 95: | ||
** {{bug|860659}} - elementslib.Lookup() fails with 'Expression \"id(\"main-window\")\" returned null. Anonymous == false"' | ** {{bug|860659}} - elementslib.Lookup() fails with 'Expression \"id(\"main-window\")\" returned null. Anonymous == false"' | ||
** {{bug|791634}} - Make waitFor() method available in the assertions module (also for soft assertions) | ** {{bug|791634}} - Make waitFor() method available in the assertions module (also for soft assertions) | ||
** {{bug|763461}} - Mozmill tests for cut-off elements should not report failure for scrollable direction | |||
** {{bug|763470}} - False positive test for cut-off elements for advanced pane tabs | |||
** {{bug|764640}} - Mozmill returns PASS/SKIP/FAIL == 0 for test_assertions.js | |||
** {{bug|860670}} - mozmill.utils.getWindowByType fails for type: Addons:Blocklist - Fails with "Window has been found." | |||
** {{bug|860662}} - controller.select() fails to select the option by value | |||
{| border="1" cellpadding="2" align="center" | {| border="1" cellpadding="2" align="center" | ||
| Line 82: | Line 114: | ||
|- | |- | ||
| {{bug|838192}} || Review || Henrik/Dave || Add screenshot functionality to assertion module (low priority with mozmill 2.0 looming) | | {{bug|838192}} || Review || Henrik/Dave || Add screenshot functionality to assertion module (low priority with mozmill 2.0 looming) | ||
|- | |- | ||
| {{bug|785014}} || Need-info || Alexandre Poirot || Build restartless.xpi as a plain bootstrapped extension | | {{bug|785014}} || Need-info || Alexandre Poirot || Build restartless.xpi as a plain bootstrapped extension | ||
| Line 98: | Line 126: | ||
|- | |- | ||
| {{bug|860665}} || Feedback || Henrik/Dave || controller.waitForPageLoad() fails when working with Private Browsing windows. | | {{bug|860665}} || Feedback || Henrik/Dave || controller.waitForPageLoad() fails when working with Private Browsing windows. | ||
|- | |||
| {{bug|827276}} || Needinfo || Alex Keybl || Test failure 'Check for updates has been completed' in /restartTests/testSoftwareUpdateAutoProxy/test2.js | |||
|} | |} | ||
| Line 133: | Line 163: | ||
==Action items== | ==Action items== | ||
* {{new|}} | * {{prev|}} Henrik - Update the goals page | ||
* {{prev|}} Anthony: Ensure any Metro tests are suitably tagged with in-testsuite? | |||
* {{new|}} Andreea: File bug about the broken elementslib handling with elements | |||