Auto-tools/Automation Development/Meetings/130204: Difference between revisions
< Auto-tools | Automation Development | Meetings
AndreeaMatei (talk | contribs) |
|||
(22 intermediate revisions by 7 users not shown) | |||
Line 14: | Line 14: | ||
= Last Weeks Action Items = | = Last Weeks Action Items = | ||
* {{ | * {{done|}}Henrik: Update goals on the wiki page | ||
* {{prev|}}Clint: Follow up on WebQA review process guidelines | ** http://publish.smartsheet.com/539082bf9b3a40509f20125aa59fa2ec | ||
* {{prev|}}Clint: Followup with Releng about the idea to report directly to tbpl | * {{prev|}} Clint: Follow up on WebQA review process guidelines | ||
* {{prev|}}Dave: Investigate and implement binding the endurance tests to specific nodes of the CI | * {{prev|}} Clint: Followup with Releng about the idea to report directly to tbpl | ||
* {{prev|}}Henrik: Check mozmill or mozmill-automation so we do not fail when we can't send the report | * {{prev|}} Dave: Investigate and implement binding the endurance tests to specific nodes of the CI | ||
* {{ | ** Pull request is available at https://github.com/mozilla/mozmill-ci/pull/200/ | ||
* {{prev|}} Henrik: Check mozmill or mozmill-automation so we do not fail when we can't send the report | |||
* {{done|}} Henrik: Start discussion thread how to get Andreea more involved | |||
= Highlights = | = Highlights = | ||
* [ | * [davehunt] I'm highlighting this every week, but https://datazilla.mozilla.org/b2g/ has been updated again to track individual branches and uses apploadtime for more accurate measurements. | ||
* [davehunt] [http://whimboo.github.com/fosdem2013/mozilla.html?full#Cover FOSDEM talk] went well, not as many questions as I'd hoped but the live demo worked! Video should be available soon at https://air.mozilla.org/ | |||
= Goals Overview = | = Goals Overview = | ||
Line 41: | Line 44: | ||
== Gaia UI (Dave) == | == Gaia UI (Dave) == | ||
* | * We had some major issues with builds at the end of last week, where remote debugging was disabled by default. Latest builds are working again. | ||
* A number of new tests have landed | |||
== WebAPI (Rob) == | == WebAPI (Rob) == | ||
Line 47: | Line 51: | ||
== WebRTC (Henrik) == | == WebRTC (Henrik) == | ||
* | * Working with Clint and Jason Smith on a proposal how to best continue | ||
= Softvision Updates = | = Softvision Updates = | ||
== Desktop Automation == | |||
* Fixed: | * Fixed: | ||
** {{bug|684679}} - Mozmill test for installing and uninstalling a blocklisted extension | ** {{bug|684679}} - Mozmill test for installing and uninstalling a blocklisted extension | ||
Line 58: | Line 64: | ||
** {{bug|830688}} - Make use of the Services.jsm module in all lib modules and tests | ** {{bug|830688}} - Make use of the Services.jsm module in all lib modules and tests | ||
** {{bug|822121}} - Test failure "There is no disk storage in use - '2101248' should equal '0'" in testPrivateBrowsing/testAboutCache.js | ** {{bug|822121}} - Test failure "There is no disk storage in use - '2101248' should equal '0'" in testPrivateBrowsing/testAboutCache.js | ||
** {{bug|600291}} - Update test module testSearchAddons to reflect the new Add-ons API | |||
* Blocked bugs: | * Blocked bugs: | ||
Line 64: | Line 71: | ||
** {{bug|763159}} - Test failure 'secure.mur.at == erle.mur.at ' in testSecurity/testUnknownIssuer.js | ** {{bug|763159}} - Test failure 'secure.mur.at == erle.mur.at ' in testSecurity/testUnknownIssuer.js | ||
* Waiting for review / Checkin needed | * Waiting for review / feedback / Checkin needed | ||
** {{bug|800872}} - Test failure "Window number '1' has been opened" in endurance//testTabbedBrowsing_OpenNewWindow/test1.js | ** {{bug|800872}} - Test failure "Window number '1' has been opened" in endurance//testTabbedBrowsing_OpenNewWindow/test1.js | ||
** {{bug|800800}} - Ensure we always wait for removeAllHistory() in places.js to stop the failure 'Browsing History has been cleared' | ** {{bug|800800}} - Ensure we always wait for removeAllHistory() in places.js to stop the failure 'Browsing History has been cleared' | ||
** {{bug| | ** {{bug|823064}} - Refactor testDownloadManagerClosed.js for the new PB per-window mode | ||
** {{bug|826645}} - Add throws() and doesNotThrow() method to the assertions module | |||
** {{bug|828185}} - Change as many as possible instances of "throw" to assert.* calls | |||
** f? {{bug|826251}} - Make testPreferences/testPreferredLanguage.js language independend | |||
** f? {{bug|827752}} - Create Mozmill test to verify that Flash content respects Firefox's private browsing | |||
** f? {{bug|489474}} - [mozmill] Write automated tests for user generated video controls | |||
** f? {{bug|732353}} - Disable all Discovery Pane tests due to unpredictable web dependencies | |||
* Working on: | * Working on: | ||
** {{bug| | ** {{bug|826693}} - Failure: controller.waitForPageLoad(): Timeout waiting for page loaded. - P1 | ||
** {{bug|834163}} - Teardown code for /testSearch tests should not exercise ui path which can cause failures like: "Modal dialog has been found and processed" | ** {{bug|834163}} - Teardown code for /testSearch tests should not exercise ui path which can cause failures like: "Modal dialog has been found and processed" | ||
** {{bug|780957}} - Failure in /testAddons_pluginDisabledAfterRestart/test2.js | Shockwave Flash is enabled - 'false' should equal 'true' | |||
** {{bug|783484}} - Test failure 'Shutdown expected but none detected before end of test ' in /restartTests/testAddons_uninstallExtension/test4.js | |||
** {{bug|790538}} - Enhance update tests to check for the 'what's new page' when `actions='silent'` given in update snippet | |||
** {{bug|813170}} - Failure in testAddons_RestartlessExtensionWorksAfterRestart/test1.js | Disconnect Error: Application unexpectedly closed | |||
*** Unable to reproduce so far with ESR10 builds on the remote XP machine. I still have about one week of testing to do | |||
** {{bug|807251}} - Test failure "Suggestions are different - got 'false'" in testSearch/testSearchSuggestions.js | |||
** {{bug|786306}} - Add more logic to the restart tests to skip following test files if a test is failing | |||
** {{bug|834632}} - Update "testPluginDisableAffectsAboutPlugins.js" so it works with current features of Firefox about:plugins | |||
== Mobile (Robocop) == | |||
*'''Automation testing status''' | |||
* Progress on [https://docs.google.com/spreadsheet/ccc?key=0Ak8KPfUd1-5udC1FRXBHMlA2MkJLOEp0NFkxTnBtS1E#gid=0 Smoke test Automation] | |||
* Bugs with work in progress: | |||
** {{Bug|820276}} - Robocop: Make editBookmark and checkBookmarkEdit methods available to other tests | |||
** {{Bug|822259}} - Robocop: Add test for Find in Page feature | |||
** {{Bug|820859}} - Robocop: Add test for 'Clear Private Data' | |||
* Bugs awaiting review: | |||
** {{Bug|745041}} - Robocop: Add test for 'Clear History' | |||
** {{Bug|830834}} - Robocop: Add test for 'Show Search Suggestions' feature | |||
** {{Bug|830755}} - Robocop: Add test for 'Settings Menu Items' feature | |||
** {{Bug|833832}} - Robocop: Add test for Import from Android feature | |||
* Bugs with accepted patches: | |||
** {{Bug|777719}} - Batch of Robocop tests | |||
** {{Bug|818390}} - Robocop: Improve the solution for identifying the device type and OS | |||
** {{Bug|747835}} - Robocop: Add automated test for Back and Forward Buttons | |||
** {{Bug|732212}} - Robocop: Add tests for doorhanger notifications | |||
** {{Bug|826261}} - Robocop: Add test for Share Link option from the Firefox Mobile Menu | |||
** {{Bug|822681}} - Robocop: Add test for 'Tabs Drawer Menu' feature | |||
** {{Bug|817970}} - Robocop: Add tests for Bookmark remove and edit from context menu | |||
** {{Bug|822256}} - Robocop: Add test for 'System Pages' feature | |||
= Other Projects = | = Other Projects = | ||
* | * [davehunt] Hoping to release initial basic version of b2gbackup today, which uses gaiatest to backup/restore data from a device | ||
= Roundtable = | = Roundtable = | ||
* | * [henrik] How can we improve the review queue handling | ||
= Presentation / Demo = | = Presentation / Demo = |
Latest revision as of 17:24, 4 February 2013
Dial in
# Vidyo: https://v.mozilla.com/flex.html?roomdirect.html&key=PGtLpx3XQGJz # Phone: 650-903-0800 or 650-215-1282 x92 Conf# 9654 (US/INTL) # 1-800-707-2533 (pin 369) Conf# 9654 (US) # IRC: irc://irc.mozilla.org:6697/#automation
General
- Attendees:
- TBD
- This meeting:
- Chair: Dave Hunt
Last Weeks Action Items
- [DONE] Henrik: Update goals on the wiki page
- [CARRY OVER] Clint: Follow up on WebQA review process guidelines
- [CARRY OVER] Clint: Followup with Releng about the idea to report directly to tbpl
- [CARRY OVER] Dave: Investigate and implement binding the endurance tests to specific nodes of the CI
- Pull request is available at https://github.com/mozilla/mozmill-ci/pull/200/
- [CARRY OVER] Henrik: Check mozmill or mozmill-automation so we do not fail when we can't send the report
- [DONE] Henrik: Start discussion thread how to get Andreea more involved
Highlights
- [davehunt] I'm highlighting this every week, but https://datazilla.mozilla.org/b2g/ has been updated again to track individual branches and uses apploadtime for more accurate measurements.
- [davehunt] FOSDEM talk went well, not as many questions as I'd hoped but the live demo worked! Video should be available soon at https://air.mozilla.org/
Goals Overview
Blockers
- The following are active blockers: http://bit.ly/Tph0NY
Issues
- bug 832510 - Merge l10n build requests for same locale
- bug 774892 - Unable to type in Firefox if it's started with focusmanager.testmode enabled on Linux
- bug 745975 - Pulse: Repack builders for Firefox candidate builds (beta/release) should send out notifications for individual locales
- bug 757122 - Pulse: Notifications for finished builds should not be send out before the builds are available for download
- bug 781129 - Pulse: Notifications for outdated builds are getting send via Pulse
- bug 777354 - Regression in shutdown time of httpd.js (~6s) when multiple local pages have been opened
Project Updates
Gaia UI (Dave)
- We had some major issues with builds at the end of last week, where remote debugging was disabled by default. Latest builds are working again.
- A number of new tests have landed
WebAPI (Rob)
- TBD
WebRTC (Henrik)
- Working with Clint and Jason Smith on a proposal how to best continue
Softvision Updates
Desktop Automation
- Fixed:
- bug 684679 - Mozmill test for installing and uninstalling a blocklisted extension
- bug 830281 - Test failure "Not enough arguments [nsIWebBrowserPersist.saveURI]" in l10n//testAccessKeys/test1.js
- bug 785041 - Mozmill test failure testSecurity/testGreyLarry.js with "Controller.assertJSProperty(ID: page-proxy-favicon)
- bug 831714 - Replace mozmill.utils.waitFor with assert/expect.waitFor in remaining Mozmill tests
- bug 835132 - Test failure "Current domain name matches target domain name" in /testSearch/testSearchSelection.js
- bug 830688 - Make use of the Services.jsm module in all lib modules and tests
- bug 822121 - Test failure "There is no disk storage in use - '2101248' should equal '0'" in testPrivateBrowsing/testAboutCache.js
- bug 600291 - Update test module testSearchAddons to reflect the new Add-ons API
- Blocked bugs:
- bug 788531 - Revise default delay for endurance test to make scenarios more realistic - needs ESX cluster
- bug 795398 - Mozmill test to check that MD5 hash signatures are no longer accepted - blocked by bug 804952 - Setup virtual host on mozqa.com for Mozmill MD5 hash signatures check
- bug 763159 - Test failure 'secure.mur.at == erle.mur.at ' in testSecurity/testUnknownIssuer.js
- Waiting for review / feedback / Checkin needed
- bug 800872 - Test failure "Window number '1' has been opened" in endurance//testTabbedBrowsing_OpenNewWindow/test1.js
- bug 800800 - Ensure we always wait for removeAllHistory() in places.js to stop the failure 'Browsing History has been cleared'
- bug 823064 - Refactor testDownloadManagerClosed.js for the new PB per-window mode
- bug 826645 - Add throws() and doesNotThrow() method to the assertions module
- bug 828185 - Change as many as possible instances of "throw" to assert.* calls
- f? bug 826251 - Make testPreferences/testPreferredLanguage.js language independend
- f? bug 827752 - Create Mozmill test to verify that Flash content respects Firefox's private browsing
- f? bug 489474 - [mozmill] Write automated tests for user generated video controls
- f? bug 732353 - Disable all Discovery Pane tests due to unpredictable web dependencies
- Working on:
- bug 826693 - Failure: controller.waitForPageLoad(): Timeout waiting for page loaded. - P1
- bug 834163 - Teardown code for /testSearch tests should not exercise ui path which can cause failures like: "Modal dialog has been found and processed"
- bug 780957 - Failure in /testAddons_pluginDisabledAfterRestart/test2.js | Shockwave Flash is enabled - 'false' should equal 'true'
- bug 783484 - Test failure 'Shutdown expected but none detected before end of test ' in /restartTests/testAddons_uninstallExtension/test4.js
- bug 790538 - Enhance update tests to check for the 'what's new page' when `actions='silent'` given in update snippet
- bug 813170 - Failure in testAddons_RestartlessExtensionWorksAfterRestart/test1.js | Disconnect Error: Application unexpectedly closed
- Unable to reproduce so far with ESR10 builds on the remote XP machine. I still have about one week of testing to do
- bug 807251 - Test failure "Suggestions are different - got 'false'" in testSearch/testSearchSuggestions.js
- bug 786306 - Add more logic to the restart tests to skip following test files if a test is failing
- bug 834632 - Update "testPluginDisableAffectsAboutPlugins.js" so it works with current features of Firefox about:plugins
Mobile (Robocop)
- Automation testing status
- Progress on Smoke test Automation
- Bugs with work in progress:
- bug 820276 - Robocop: Make editBookmark and checkBookmarkEdit methods available to other tests
- bug 822259 - Robocop: Add test for Find in Page feature
- bug 820859 - Robocop: Add test for 'Clear Private Data'
- Bugs awaiting review:
- bug 745041 - Robocop: Add test for 'Clear History'
- bug 830834 - Robocop: Add test for 'Show Search Suggestions' feature
- bug 830755 - Robocop: Add test for 'Settings Menu Items' feature
- bug 833832 - Robocop: Add test for Import from Android feature
- Bugs with accepted patches:
- bug 777719 - Batch of Robocop tests
- bug 818390 - Robocop: Improve the solution for identifying the device type and OS
- bug 747835 - Robocop: Add automated test for Back and Forward Buttons
- bug 732212 - Robocop: Add tests for doorhanger notifications
- bug 826261 - Robocop: Add test for Share Link option from the Firefox Mobile Menu
- bug 822681 - Robocop: Add test for 'Tabs Drawer Menu' feature
- bug 817970 - Robocop: Add tests for Bookmark remove and edit from context menu
- bug 822256 - Robocop: Add test for 'System Pages' feature
Other Projects
- [davehunt] Hoping to release initial basic version of b2gbackup today, which uses gaiatest to backup/restore data from a device
Roundtable
- [henrik] How can we improve the review queue handling
Presentation / Demo
- [%name%] %topic%
Triage
Personal Status
For the personal status please check the weekly status updates:
Meeting Notes
The meeting notes for this meeting can be read in the following etherpad:
Action items
- [NEW] %name%: %task%