Releases/Firefox 25/Test Plan: Difference between revisions
(572 intermediate revisions by 14 users not shown) | |||
Line 3: | Line 3: | ||
= Firefox 25 Test Plan = | = Firefox 25 Test Plan = | ||
== Summary == | == Summary == | ||
{| class="fullwidth-table" | The following is the test plan for Firefox 25 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 25. After the release this document will be used as an archive of what was done to validate this release. | ||
{| class="querytable fullwidth-table" | |||
|- | |- | ||
| '''Status''' | | '''Status''' | ||
Line 10: | Line 12: | ||
| '''Lead''' | | '''Lead''' | ||
| [mailto:ahughes@mozilla.com Anthony Hughes] (irc: ashughes) | | [mailto:ahughes@mozilla.com Anthony Hughes] (irc: ashughes) | ||
|- | |- | ||
| '''Builds''' | | '''Builds''' | ||
Line 18: | Line 17: | ||
|} | |} | ||
; ''If you have some free time, please pick a task below and get in touch with one of the leads.'' | |||
''If you have some free time, please pick a task below and get in touch with one of the leads.'' | |||
== Schedule == | == Schedule == | ||
Line 41: | Line 38: | ||
|} | |} | ||
== | == Aurora == | ||
=== Branch Migration === | |||
* All | ==== Pre Sign-off Tasks ==== | ||
* | * {{done|}} Firefox branch migrated (Release Management) | ||
* | * {{done|}} Mozmill Tests branch migrated (Automation Development) | ||
* 0 | * {{done|}} Updates throttled on ''aurora'' channel (Release Engineering) | ||
* 0 | * {{done|}} Updates verified throttled on ''aurora'' channel (QA) | ||
==== Sign-off Tasks ==== | |||
* {{done|}} Mozmill Daily ''functional'' testruns pass with no unexpected failures/regressions ([http://mozmill-daily.blargon7.com/#/functional/reports?branch=25.0&platform=All&from=2013-08-05&to=2013-08-09 report]) | |||
* {{done|}} Mozmill Daily ''l10n'' testruns pass with no unexpected failures/regressions ([http://mozmill-daily.blargon7.com/#/l10n/reports?branch=25.0&platform=All&from=2013-08-05&to=2013-08-09 report]) | |||
* {{done|}} Mozmill Daily ''endurance'' testruns pass with no unexpected failures/regressions ([http://mozmill-daily.blargon7.com/#/endurance/charts?branch=25.0&platform=All&from=2013-07-11&to=2013-08-09 report]) | |||
* {{done|}} Mozmill Daily ''remote'' testruns pass with no unexpected failures/regressions ([http://mozmill-daily.blargon7.com/#/remote/reports?branch=25.0&platform=All&from=2013-08-05&to=2013-08-09 report]) | |||
* {{done|}} Mozmill Daily ''auroratest update'' testruns pass with no unexpected failures/regressions ([http://mozmill-daily.blargon7.com/#/update/detail?branch=25.0&channel=auroratest&from=2013-08-05&to=2013-08-09&target=25.0a2 report]) | |||
==== Post Sign-off Tasks ==== | |||
* {{done|}} Aurora sign-off email sent (QA) | |||
* {{done|}} Updates unthrottled on ''aurora'' channel (Release Engineering) | |||
* {{done|}} Updates verified unthrottled on ''aurora'' channel (QA) | |||
* {{miss|}} Verify Stub Installers stage the new Aurora build ({{bug|910442}}) | |||
=== Pre-Beta Sign-off === | |||
==== Checklist ==== | |||
* {{done|}} Mozmill-CI automation testruns checked for potential blocking regressions | |||
* {{done|}} Features signed-off for Beta readiness, verified disabled if not | |||
* {{ok|}} WebRTC spotchecked for potential blocking regressions, parity with Firefox 22 | |||
* {{ok|}} Flagged bug fixes verified | |||
==== Automation ==== | |||
The following tracks the results of daily automation run against the Aurora builds as indicated by the Mozmill [http://mozmill-daily.blargon7.com/#/addons/reports?branch=25.0&platform=All&from=2013-08-04&to=2013-09-16 Addons], [http://mozmill-daily.blargon7.com/#/endurance/charts?branch=25.0&platform=All&from=2013-08-04&to=2013-09-16 Endurance], [http://mozmill-daily.blargon7.com/#/functional/top?branch=25.0&platform=All&from=2013-08-04&to=2013-09-16 Functional], [http://mozmill-daily.blargon7.com/#/l10n/reports?branch=25.0&platform=All&from=2013-08-04&to=2013-09-16 L10n], [http://mozmill-daily.blargon7.com/#/remote/reports?branch=25.0&platform=All&from=2013-08-04&to=2013-09-16 Remote], and [http://mozmill-daily.blargon7.com/#/update/detail?branch=All&channel=aurora&from=2013-08-04&to=2013-09-16&target=25.0a2 Update] reports. | |||
{| class="querytable" | |||
| class="header" | Top Failures | |||
| class="header" | Platform | |||
| class="header" | Instances | |||
| class="header" | Bug | |||
| class="header" | Status | |||
|- | |||
| /testSecurity/testGreenLarry.js | |||
| All | |||
| 130 | |||
| {{bug|907101}} | |||
| <span style="color:green">'''FIXED'''</span> (2013-08-21) | |||
|- | |||
| /testPreferences/testRestoreHomepageToDefault.js | |||
| All | |||
| 41 | |||
| {{bug|912363}} | |||
| <span style="color:grey">'''DISABLED'''</span> (2013-09-05) | |||
|- | |||
| /testPopups/testPopupsAllowed.js | |||
| All | |||
| 36 | |||
| {{bug|790218}} | |||
| <span style="color:green">'''FIXED'''</span> (2013-09-09) | |||
|- | |||
| /testSecurity/testSafeBrowsingNotificationBar.js | |||
| All | |||
| 32 | |||
| {{bug|905033}} | |||
| <span style="color:grey">'''DISABLED'''</span> (2013-08-12) | |||
|- | |||
| /testFindInPage/testFindInPage.js | |||
| All | |||
| 24 | |||
| {{bug|909227}} | |||
| <span style="color:red">'''BLOCKED'''</span> by l10n {{bug|913396}} (2013-09-05) | |||
|- | |||
| /testSecurity/testDVCertificate.js | |||
| All | |||
| 24 | |||
| {{bug|890487}} | |||
| <span style="color:green">'''FIXED'''</span> (2013-08-06) | |||
|- | |||
| /testSecurity/testMixedContentPage.js | |||
| All | |||
| 24 | |||
| {{bug|901951}} | |||
| <span style="color:green">'''FIXED'''</span> (2013-08-06) | |||
|- | |||
| /testPrivateBrowsing/testAboutCache.js | |||
| Windows | |||
| 23 | |||
| {{bug|888924}} | |||
| '''ASSIGNED''' (2013-09-05) | |||
|- | |||
| /testPopups/testPopupsBlocked.js | |||
| Windows | |||
| 13 | |||
| n/a | |||
| '''WORKSFORME''' (2013-09-04) | |||
|} | |||
{| class="querytable" | |||
| class="header" | Resident Memory Usage | |||
| class="header" | Week 1 | |||
| class="header" | Week 2 | |||
| class="header" | Week 3 | |||
| class="header" | Week 4 | |||
| class="header" | Week 5 | |||
| class="header" | Week 6 | |||
| class="header" | Net | |||
|- | |||
| Windows XP | |||
| 105 MB | |||
| 106 MB | |||
| 106 MB | |||
| 106 MB | |||
| 106 MB | |||
| 104 MB | |||
| <span style="color:green">-0.952%</span> | |||
|- | |||
| Windows Vista | |||
| 97 MB | |||
| 97 MB | |||
| 97 MB | |||
| 97 MB | |||
| 97 MB | |||
| 97 MB | |||
| no change | |||
|- | |||
| Windows 7 32-bit | |||
| 100 MB | |||
| 100 MB | |||
| 100 MB | |||
| 100 MB | |||
| 100 MB | |||
| 99 MB | |||
| <span style="color:green">-1.000%</span> | |||
|- | |||
| Windows 7 64-bit | |||
| 102 MB | |||
| 102 MB | |||
| 102 MB | |||
| 102 MB | |||
| 102 MB | |||
| 101 MB | |||
| <span style="color:green">-0.980%</span> | |||
|- | |||
| Windows 8 32-bit | |||
| 130 MB | |||
| 124 MB | |||
| 122 MB | |||
| 123 MB | |||
| 122 MB | |||
| 102 MB | |||
| <span style="color:green">-21.538%</span> | |||
|- | |||
| Windows 8 64-bit | |||
| 132 MB | |||
| 126 MB | |||
| 123 MB | |||
| 124 MB | |||
| 124 MB | |||
| 104 MB | |||
| <span style="color:green">-21.212%</span> | |||
|- | |||
| Mac OSX 10.6 | |||
| 138 MB | |||
| 134 MB | |||
| 134 MB | |||
| 134 MB | |||
| 134 MB | |||
| 132 MB | |||
| <span style="color:green">-4.348%</span> | |||
|- | |||
| Mac OSX 10.7 | |||
| 203 MB | |||
| 200 MB | |||
| 200 MB | |||
| 201 MB | |||
| 199 MB | |||
| 197 MB | |||
| <span style="color:green">-2.956%</span> | |||
|- | |||
| Mac OSX 10.8 | |||
| 195 MB | |||
| 189 MB | |||
| 190 MB | |||
| 190 MB | |||
| 191 MB | |||
| 192 MB | |||
| <span style="color:green">-1.538%</span> | |||
|- | |||
| Ubuntu LTS 32-bit | |||
| 103 MB | |||
| 100 MB | |||
| 100 MB | |||
| 100 MB | |||
| 98 MB | |||
| 98 MB | |||
| <span style="color:green">-4.854%</span> | |||
|- | |||
| Ubuntu LTS 64-bit | |||
| 135 MB | |||
| 131 MB | |||
| 131 MB | |||
| 131 MB | |||
| 131 MB | |||
| 129 MB | |||
| <span style="color:green">-4.444%</span> | |||
|- | |||
| Ubuntu Desktop 32-bit | |||
| 102 MB | |||
| 100 MB | |||
| 100 MB | |||
| 100 MB | |||
| 101 MB | |||
| 99 MB | |||
| <span style="color:green">-2.941%</span> | |||
|- | |||
| Ubuntu Desktop 64-bit | |||
| 135 MB | |||
| 131 MB | |||
| 131 MB | |||
| 131 MB | |||
| 132 MB | |||
| 130 MB | |||
| <span style="color:green">-3.704%</span> | |||
|- | |||
|} | |||
==== Features ==== | |||
The following features are tracked and tested as indicated by the ''[https://www-dev.allizom.org/en-US/firefox/25.0a2/auroranotes/ release notes]'', ''[https://bugzilla.mozilla.org/buglist.cgi?list_id=7759236&o1=equals&v1=25%2B&f1=cf_tracking_firefox_relnote&query_format=advanced&product=Firefox relnote+]'', ''[http://mzl.la/17CujPp Firefox]'', and ''[http://mzl.la/17xI1oy Platform]'' feature bugs. | |||
* {{done|}} [https://bugzilla.mozilla.org/show_bug.cgi?id=779297 Web Audio] (Manuela Muntean) | |||
* {{done|}} [https://bugzilla.mozilla.org/show_bug.cgi?id=565552 Find Bar Redesign] (Manuela Muntean) | |||
* {{done|}} [https://bugzilla.mozilla.org/show_bug.cgi?id=833943 Reset Firefox Saves Windows and Tabs] (Ioana Budnar) | |||
* {{done|}} [https://bugzilla.mozilla.org/show_bug.cgi?id=498181 Reset Profile When Unused for Months] (Simona Marcu) | |||
* {{done|}} [https://bugzilla.mozilla.org/show_bug.cgi?id=483446 CSS3 background-attachment:local Support] (Petruta Rasa) | |||
* {{done|}} [https://bugzilla.mozilla.org/show_bug.cgi?id=885366 Larry Panel] (Bogdan Maris) | |||
* {{done|}} [https://bugzilla.mozilla.org/show_bug.cgi?id=866847 Emacs Map#forEach and Set#forEach] (Paul Silaghi) | |||
* {{done|}} [https://bugzilla.mozilla.org/show_bug.cgi?id=717379 ES6 math functions] (Mihai Morar) | |||
* {{done|}} [https://bugzilla.mozilla.org/show_bug.cgi?id=802895 Support in-line HTML5 iframes] (Alexandra Lucinet) | |||
* {{done|}} [https://bugzilla.mozilla.org/show_bug.cgi?id=910167 Session Restore backend work, tab close & restore] (Cornel Ionce) | |||
* {{drop|}} [https://bugzilla.mozilla.org/show_bug.cgi?id=870100 Background Thumbnail Service] (Paul Silaghi) | |||
* {{drop|}} [https://bugzilla.mozilla.org/show_bug.cgi?id=604039 Prototype DOM Gamepad API] (Alexandra Lucinet) | |||
==== WebRTC ==== | |||
Using the [https://moztrap.mozilla.org/manage/case/9523/ Moztrap test] as a guideline, verify that Firefox 25.0a2 is no worse than Firefox 22.0 when making AppRTC calls of >5 minutes in length. When testing, here are the guidelines to follow: | |||
* Have only one browser open on each machine at any given time | |||
* Have only one call running at any given time on a particular machine | |||
* Make sure the caller and callee are always on *different* machines. | |||
* Please don't test any 3+-way calls for this sanity check -- We just want to see the results for 1:1 (basic) calling | |||
* If you find regressions, [http://mzl.la/16pA4TW report a bug] and CC Randell Jesup and Maire Reavy; they can help track down if the regressions are real or not | |||
** ''Tip'': be sure to provide extremely detailed steps to reproduce and witnessed results, as well as detailed information about your test environment; more information is better than not enough information. | |||
{| class="querytable" | |||
| class="header" | Caller | |||
| class="header" | Callee | |||
| class="header" | Result | |||
|- | |||
| Firefox 25.0a2 on Windows 7 | |||
| Firefox 25.0a2 on Windows 7 | |||
| '''PASS''' | |||
|- | |||
| Firefox 22.0 on Windows 7 | |||
| Firefox 25.0a2 on Windows XP | |||
| '''FAIL (1*)''' | |||
|- | |||
| Chrome Stable on Windows 7 | |||
| Firefox 25.0a2 on Windows 8 | |||
| '''PASS''' | |||
|- | |||
| Firefox 25.0a2 on Windows 7 | |||
| Firefox 22.0 on Mac OSX 10.8 | |||
| '''PASS (2*)''' | |||
|- | |||
| Firefox 25.0a2 on Windows 7 | |||
| Chrome Stable on Ubuntu 12.04 | |||
| '''PASS''' | |||
|- | |||
| Firefox 22.0 on Windows XP | |||
| Firefox 25.0a2 on Windows 7 | |||
| '''FAIL (4*)''' | |||
|- | |||
| Firefox 25.0a2 on Windows XP | |||
| Firefox 22.0 on Windows XP | |||
| '''FAIL (10*)''' | |||
|- | |||
| Firefox 25.0a2 on Windows XP | |||
| Chrome Stable on Windows 8 | |||
| '''FAIL (5*)''' | |||
|- | |||
| Chrome Stable on Windows XP | |||
| Firefox 25.0a2 on Mac OSX 10.8 | |||
| '''PASS''' | |||
|- | |||
| Firefox 22.0 on Windows XP | |||
| Firefox 25.0a2 on Ubuntu 13.04 | |||
| '''FAIL (10*)''' | |||
|- | |||
| Chrome Stable on Windows 8 | |||
| Firefox 25.0a2 on Windows 7 | |||
| '''FAIL (8*)''' | |||
|- | |||
| Firefox 25.0a2 on Windows 8 | |||
| Chrome Stable on Windows XP | |||
| '''FAIL (7*)''' | |||
|- | |||
| Firefox 22.0 on Windows 8 | |||
| Firefox 25.0a2 on Windows 8 | |||
| '''PASS''' | |||
|- | |||
| Firefox 25.0a2 on Windows 8 | |||
| Firefox 22.0 on Windows 8 | |||
| '''PASS''' | |||
|- | |||
| Firefox 22.0 on Windows 8 | |||
| Firefox 25.0a2 on Mac OSX 10.8 | |||
| '''FAIL (6*)''' | |||
|- | |||
| Chrome Stable on Windows 8 | |||
| Firefox 25.0a2 on Ubuntu 13.04 | |||
| '''PASS''' | |||
|- | |||
| Firefox 25.0a2 on Mac OSX 10.8 | |||
| Firefox 22.0 on Windows 7 | |||
| '''FAIL (9*)''' | |||
|- | |||
| Chrome Stable on Mac OSX 10.8 | |||
| Firefox 25.0a2 on Windows XP | |||
| '''FAIL (9*) | |||
|- | |||
| Firefox 22.0 on Mac OSX 10.8 | |||
| Firefox 25.0a2 on Windows 8 | |||
| '''FAIL (11*)''' | |||
|- | |||
| Firefox 25.0a2 on Mac OSX 10.8 | |||
| Chrome Stable on Mac OSX 10.8 | |||
| '''PASS''' | |||
|- | |||
| Firefox 25.0a2 on Mac OSX 10.8 | |||
| Firefox 22.0 on Ubuntu 13.04 | |||
| '''FAIL (9*)''' | |||
|- | |||
| Firefox 25.0a2 on Ubuntu 12.04 | |||
| Chrome Stable on Windows 7 | |||
| '''PASS''' | |||
|- | |||
| Firefox 22.0 on Ubuntu 12.04 | |||
| Firefox 25.0a2 on Windows XP | |||
| '''PASS''' | |||
|- | |||
| Chrome Stable on Ubuntu 12.04 | |||
| Firefox 25.0a2 on Windows 8 | |||
| '''PASS''' | |||
|- | |||
| Firefox 25.0a2 on Ubuntu 12.04 | |||
| Firefox 22.0 on Mac OSX 10.8 | |||
| '''PASS''' | |||
|- | |||
| Firefox 25.0a2 on Ubuntu 12.04 | |||
| Chrome Stable on Ubuntu 13.04 | |||
| '''PASS''' | |||
|- | |||
|} | |||
Detailed Results | |||
* (1*) The caller hears himself, as well. The callee hears delayed sound (1-2sec) after a few minutes | |||
** not reproducible if caller uses latest Aurora 25.0a2. | |||
* (2*) The caller got a crash after ~12 min, while opening a page in a new tab: https://crash-stats.mozilla.com/report/index/97cf87a4-284f-4cd0-9a17-ed6cf2130906 | |||
** one time occurrence, not related to WebRTC | |||
* (4*) Both caller and callee hear delayed sound (1-2secs) - reproducible also if callee uses Firefox 23.0.1 | |||
* (5*) The caller hears delayed sound | |||
** reproducible also with FF23.0.1 and 24b9; moreover, with 23.0.1 and 24b.9, the delay appears on both sides | |||
* (6*) The callee hear himself, as well | |||
** reproducible also with 23.0.1 and Firefox 24 beta 10 | |||
* (7*) Callee's conversation page crashes when caller attends | |||
** reproducible with FF23.0.1 or 24b9 (used by caller) | |||
* (8*) when caller connects, both attendances remain stuck in “Connecting…” mode | |||
** working with 23.0.1 and 24b9 | |||
* (9*) when caller connects, both attendances remain stuck in “Connecting…” mode | |||
** working with 23.0.1 and 24b9 | |||
** the problem is the TURN server which is giving a bogus address | |||
* (10*) sometimes, the image and sound is choppy and there is a delayed sound: ~5 seconds at caller, ~10seconds at callee; callee hears himself, as well | |||
** reproducible with 23.0.1 | |||
* (11*) attendants see themselves instead of seeing the other person in the call and there is no sound | |||
** reproducible with 23.0.1 | |||
==== Bug Fix Verifications ==== | |||
The following bugs have been fixed and flagged for verification. At a minimum, any critical/blocker bug should be verified '''before''' migration to Beta. | |||
<bugzilla> | |||
{ | |||
"f1":"op_sys", | |||
"o1":"nowordssubstr", | |||
"v1":"Android Gonk", | |||
"f2":"keywords", | |||
"o2":"nowordssubstr", | |||
"v2":"assertion intermittent", | |||
"f3":"component", | |||
"o3":"nowordssubstr", | |||
"v3":"Build%20Config", | |||
"f4":"product", | |||
"o4":"nowordssubstr", | |||
"v4":"QA mozilla SDK Boot2Gecko Testing", | |||
"f5":"status_whiteboard", | |||
"o5":"nowordssubstr", | |||
"v5":"qa asan Leo", | |||
"keywords":"verifyme", | |||
"cf_status_firefox25":"fixed" | |||
} | |||
</bugzilla> | |||
==== Potential Blockers ==== | |||
The following bugs have been nominated to block this milestone from uplifting to Beta. | |||
* ''no blockers currently'' | |||
== Beta 1 == | |||
=== Build Info === | |||
* Builds: [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0b1-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0b1-candidates/build1/ HTTP] | |||
* Build ID: 20130917123208 | |||
* [https://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=260fcd9e23a3&tochange=143e8c67cf1b Change Log] | |||
* [https://wiki.mozilla.org/QA/Desktop_Firefox/Releases/Configs/Fx25b1 Mozmill Configs] | |||
=== Check List === | |||
{| class="querytable" | |||
| class="header" | Criteria | |||
| class="header" | Description | |||
| class="header" | Status | |||
|- | |||
| Automation | |||
| No unexpected test failures from Mozmill functional testruns | |||
| <span style="color:green">'''SIGNED-OFF'''</span> ([http://mozmill-release.blargon7.com/#/functional/reports?branch=25.0&platform=All&from=2013-09-17&to=2013-09-18 report]) | |||
|- | |||
| Regression Testing | |||
| No unexpected failures from manually smoketesting past regressions and new features | |||
| <span style="color:green">'''SIGNED-OFF'''</span> ([https://moztrap.mozilla.org/results/cases/?filter-run=2262 report]) | |||
|- | |||
| Bug Fix Verifications | |||
| No unexpected failures from manually verifying landed bug fixes | |||
| <span style="color:green">'''SIGNED-OFF'''</span> ([https://wiki.mozilla.org/Releases/Firefox_25/Test_Plan#Verifications report]) | |||
|- | |||
| Stability Verification | |||
| No unexpected spikes in top crash report for this milestone | |||
| <span style="color:green">'''SIGNED-OFF'''</span> ([https://wiki.mozilla.org/Releases/Firefox_25/Test_Plan#Top_Crashes report]) | |||
|- | |||
| Updates on ''betatest'' channel | |||
| No unexpected failures from Mozmill update testruns on the ''betatest'' channel | |||
| <span style="color:green">'''SIGNED-OFF'''</span> ([http://mozmill-release.blargon7.com/#/update/overview?branch=25.0&channel=betatest&from=2013-09-17&to=2013-09-19&target=25.0 report]) | |||
|- | |||
| Updates on ''releasetest'' channel | |||
| No unexpected failures from Mozmill update testruns on the ''releasetest'' channel | |||
| <span style="color:green">'''SIGNED-OFF'''</span> ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=releasetest&from=2013-09-19&to=2013-09-20&target=25.0 report]) | |||
|- | |||
| Updates on ''betatest'' channel | |||
| No unexpected failures from Mozmill update testruns on the ''beta'' channel | |||
| <span style="color:green">'''SIGNED-OFF'''</span> ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=beta&from=2013-09-19&to=2013-09-20&target=25.0 report]) | |||
|} | |||
=== Regression Testing === | |||
The following areas have been identified for regression testing and will be covered via Moztrap smoketests. | |||
; Test Coverage | |||
* Breakpad | |||
* NVidia 3D-Vision | |||
* WebRTC AppRTC 1:1 calling (subset of Aurora testing) | |||
* [https://bugzilla.mozilla.org/show_bug.cgi?id=282097 DefaultKeyBinding.dict support on Mac OSX] (Paul Silaghi) | |||
* [https://bugzilla.mozilla.org/show_bug.cgi?id=779297 Web Audio] (Manuela Muntean) | |||
* [https://bugzilla.mozilla.org/show_bug.cgi?id=565552 Find Bar Redesign] (Manuela Muntean) | |||
* [https://bugzilla.mozilla.org/show_bug.cgi?id=833943 Reset Firefox Saves Windows and Tabs] (Ioana Budnar) | |||
* [https://bugzilla.mozilla.org/show_bug.cgi?id=498181 Reset Profile When Unused for Months] (Simona Marcu) | |||
* [https://bugzilla.mozilla.org/show_bug.cgi?id=885366 Larry Panel] (Bogdan Maris) | |||
* [https://bugzilla.mozilla.org/show_bug.cgi?id=910167 Session Restore backend work, tab close & restore] (Cornel Ionce) | |||
; Platform Coverage | |||
* Windows XP | |||
* Windows 7 64-bit | |||
* Windows 8.1 Preview 64-bit | |||
* Mac OSX 10.8 | |||
* Ubuntu 13.04 64-bit | |||
; Instructions | |||
* [https://moztrap.mozilla.org/runtests/environment/2262/ Run these tests] | |||
* Select one of the platforms | |||
* Follow the steps in each of the tests and report your results within Moztrap | |||
* When you get to the ''WebRTC'' smoketest refer to the ''Special Coverage for WebRTC'' section below for coverage details | |||
* If you encounter a bug, [http://mzl.la/1eM3osi report it] and add it to the [[#Issues|Issues]] list below using the {{bug|number}} template | |||
; Special Coverage for WebRTC | |||
* ''Caller Firefox/Platform -> Callee Firefox/Platform'' | |||
* Firefox 25.0b1 on Windows XP -> Firefox 25.0b1 on Windows 7 | |||
* Firefox 25.0b1 on Windows 7 -> Firefox 25.0b1 on Windows 8 | |||
* Firefox 25.0b1 on Windows 8 -> Firefox 25.0b1 on Mac OSX 10.8 | |||
* Firefox 25.0b1 on Mac OSX 10.8 -> Firefox 25.0b1 on Ubuntu 13.04 | |||
* Firefox 25.0b1 on Ubuntu 13.04 -> Firefox 25.0b1 on Windows XP | |||
=== Issues === | |||
List any new issues found or old issues reopened using the {{bug|number}} template. These issues will be triaged and escalated by the release leads. | |||
{| class="querytable" | |||
| class="header" | Bug | |||
| class="header" | Status | |||
|- | |||
| {{bug|917818}} - Findbar buttons remain in pressed state on Linux | |||
| NEW - ''does not block 25b1'' | |||
|- | |||
| {{bug|912975}} - Windows and tabs are not always restored after Reseting Firefox | |||
| NEW - ''does not block 25b1'' | |||
|- | |||
| {{bug|860249}} - [Mozmill] Exception "Sorry, cannot connect to jsbridge extension, port 24242" | |||
| NEW - ''does not block 25b1'' | |||
|- | |||
| {{bug|791195}} - Session is not restored after Firefox crash | |||
| NEW - ''does not block 25b1'' | |||
|- | |||
|} | |||
== Beta 2 == | |||
=== Build Info === | |||
* Builds: [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0b2-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0b2-candidates/build1/ HTTP] | |||
* Build ID: 20130923194050 | |||
* [https://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=143e8c67cf1b&tochange=7d23197d995a Change Log] | |||
* [https://wiki.mozilla.org/QA/Desktop_Firefox/Releases/Configs/Fx25b2 Mozmill Configs] | |||
=== Check List === | |||
{| class="querytable" | {| class="querytable" | ||
| class="header" | | | class="header" | Criteria | ||
| class="header" | | | class="header" | Description | ||
| class="header" | Status | | class="header" | Status | ||
|- | |- | ||
| | | Automation | ||
| | | No unexpected test failures from Mozmill functional testruns | ||
| [https://wiki.mozilla.org/ | | <span style="color:green">'''SIGNED-OFF'''</span> ([http://mozmill-release.blargon7.com/#/functional/reports?branch=25.0&platform=All&from=2013-09-23&to=2013-09-24 report]) | ||
| [https:// | |- | ||
| {{done|}} | | Regression Testing | ||
| No unexpected failures from manually smoketesting past regressions and new features | |||
| <span style="color:green">'''SIGNED-OFF'''</span> ([https://moztrap.mozilla.org/results/cases/?filter-run=2307 report]) | |||
|- | |||
| Bug Fix Verifications | |||
| No unexpected failures from manually verifying landed bug fixes | |||
| <span style="color:green">'''SIGNED-OFF'''</span> ([https://wiki.mozilla.org/Releases/Firefox_25/Test_Plan#Verifications report]) | |||
|- | |||
| Stability Verification | |||
| No unexpected spikes in top crash report for this milestone | |||
| <span style="color:green">'''SIGNED-OFF'''</span> ([https://wiki.mozilla.org/Releases/Firefox_25/Test_Plan#Top_Crashes report]) | |||
|- | |||
| Updates on ''betatest'' channel | |||
| No unexpected failures from Mozmill update testruns on the ''betatest'' channel | |||
| <span style="color:green">'''SIGNED-OFF'''</span> ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=betatest&from=2013-09-23&to=2013-09-24&target=25.0 report]) | |||
|- | |||
| Updates on ''releasetest'' channel | |||
| No unexpected failures from Mozmill update testruns on the ''releasetest'' channel | |||
| <span style="color:green">'''SIGNED-OFF'''</span> ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=releasetest&from=2013-09-23&to=2013-09-25&target=25.0 report]) | |||
|- | |||
| Updates on ''beta'' channel | |||
| No unexpected failures from Mozmill update testruns on the ''beta'' channel | |||
| <span style="color:green">'''SIGNED-OFF'''</span> ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=beta&from=2013-09-24&to=2013-09-25&target=25.0 report]) | |||
|} | |||
=== Regression Testing === | |||
The following areas have been identified for regression testing and will be covered via Moztrap smoketests. | |||
; Test Coverage | |||
* Breakpad | |||
* MD5 Hash De-support | |||
* Plug-in Compatibility | |||
* Exploratory testing around Flash 11.9 Beta : [https://etherpad.mozilla.org/Fx25b2-Flash11-9 etherpad] | |||
; Platform Coverage | |||
* Windows Vista | |||
* Windows 7 32-bit | |||
* Windows 8 64-bit | |||
* Mac OSX 10.7 | |||
* Ubuntu 12.04 64-bit | |||
; Instructions | |||
* [https://moztrap.mozilla.org/runtests/environment/2307/ Run these tests] | |||
* Select one of the platforms | |||
* Follow the steps in each of the tests and report your results within Moztrap | |||
* If you encounter a bug, [http://mzl.la/1eM3osi report it] and add it to the [[#Issues|Issues]] list below using the {{bug|number}} template | |||
== Beta 3 == | |||
=== Build Info === | |||
* Builds: [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0b3-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0b3-candidates/build1/ HTTP] | |||
* Build ID: 20130926170421 | |||
* [https://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=7d23197d995a&tochange=637bba8205e6 Change Log] | |||
* [https://wiki.mozilla.org/QA/Desktop_Firefox/Releases/Configs/Fx25b3 Mozmill Configs] | |||
=== Check List === | |||
* {{done|}} Automation ([http://mozmill-release.blargon7.com/#/functional/reports?branch=25.0&platform=All&from=2013-09-26&to=2013-09-27 report]) | |||
* {{done|}} Regression Testing ([https://moztrap.mozilla.org/results/cases/?filter-run=2331 report]) | |||
* {{done|}} Updates on ''betatest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=betatest&from=2013-09-27&to=2013-09-28&target=25.0 report]) | |||
* {{done|}} Updates on ''releasetest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=releasetest&from=2013-09-27&to=2013-09-28&target=25.0 report]) | |||
* {{done|}} Updates on ''beta'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=beta&from=2013-09-27&to=2013-09-28&target=25.0 report]) | |||
=== Regression Testing === | |||
The following areas have been identified for regression testing and will be covered via Moztrap smoketests. | |||
* '''Focus''': Breakpad, Web Audio, WebGL | |||
* '''Platforms''': Windows XP, Windows 7 64-bit, Windows 8.1 32-bit, Mac OSX 10.8, Ubuntu 13.04 64-bit | |||
* '''Instructions''': [https://moztrap.mozilla.org/runtests/environment/2331/ Run these tests] and [http://mzl.la/1eM3osi report bugs here]; add any issues found to the Issues Found section below | |||
== Beta 4 == | |||
=== Build Info === | |||
* Builds: [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0b4-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0b4-candidates/build1/ HTTP] | |||
* Build ID: 20131001024718 | |||
* [https://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=637bba8205e6&tochange=dd1489d534ab Change Log] | |||
* [https://wiki.mozilla.org/QA/Desktop_Firefox/Releases/Configs/Fx25b4 Mozmill Configs] | |||
=== Check List === | |||
* {{done|}} Automation ([http://mozmill-release.blargon7.com/#/functional/reports?branch=25.0&platform=All&from=2013-09-30&to=2013-10-01 report]) | |||
* {{done|}} Regression Testing ([https://moztrap.mozilla.org/results/cases/?filter-run=2343 report]) | |||
* {{done|}} Updates on ''betatest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=betatest&from=2013-10-01&to=2013-10-02&target=25.0 report]) | |||
* {{done|}} Updates on ''releasetest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=releasetest&from2013-10-01&to=2013-10-02&target=25.0 report]) | |||
* {{done|}} Updates on ''beta'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=beta&from=2013-10-01&to=2013-10-02&target=25.0 report]) | |||
=== Regression Testing === | |||
The following areas have been identified for regression testing and will be covered via Moztrap smoketests. | |||
* '''Focus''': Breakpad, Web Audio, WebGL | |||
* '''Platforms''': Windows XP, Windows Vista, Windows 8 64-bit, Mac OSX 10.7, Ubuntu 12.04 64-bit | |||
* '''Instructions''': [https://moztrap.mozilla.org/runtests/environment/2343/ Run these tests] and [http://mzl.la/1eM3osi report bugs here]; add any issues found to the Issues Found section below | |||
== Beta 6 == | |||
=== Build Info === | |||
* Builds: [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0b6-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0b6-candidates/build1/ HTTP] | |||
* Build ID: 20131007213254 | |||
* [https://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=dd1489d534ab&tochange=373f0e83dc36 Change Log] | |||
* [https://wiki.mozilla.org/QA/Desktop_Firefox/Releases/Configs/Fx25b6 Mozmill Configs] | |||
=== Check List === | |||
* {{done|}} Automation ([http://mozmill-release.blargon7.com/#/functional/reports?branch=25.0&platform=All&from=2013-10-08&to=2013-10-09 report]) | |||
* {{done|}} Regression Testing ([https://moztrap.mozilla.org/results/cases/?filter-run=2381 report]) | |||
* {{done|}} Updates on ''betatest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=betatest&from=2013-10-08&to=2013-10-09&target=25.0 report]) | |||
* {{done|}} Updates on ''releasetest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=releasetest&from2013-10-08&to=2013-10-09&target=25.0 report]) | |||
* {{done|}} Updates on ''beta'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=beta&from=2013-10-08&to=2013-10-09&target=25.0 report]) | |||
=== Regression Testing === | |||
The following areas have been identified for regression testing and will be covered via Moztrap smoketests. | |||
* '''Focus''': Breakpad, Web Audio, WebGL, TLS/SSL | |||
* '''Platforms''': Windows XP, Windows Vista, Windows 8 64-bit, Mac OSX 10.7, Ubuntu 12.04 64-bit | |||
* '''Instructions''': [https://moztrap.mozilla.org/runtests/environment/2381/ Run these tests] and [http://mzl.la/1eM3osi report bugs here]; add any issues found to the Issues Found section below | |||
* '''Exploratory testing''': [https://etherpad.mozilla.org/Fx25b6-TLS-SSL TLS/SSL etherpad] | |||
== Beta 7 == | |||
=== Build Info === | |||
* Builds: [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0b7-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0b7-candidates/build1/ HTTP] | |||
* Build ID: 20131010180222 | |||
* [https://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=373f0e83dc36&tochange=8b2c180a7b5d Change Log] | |||
* [https://wiki.mozilla.org/QA/Desktop_Firefox/Releases/Configs/Fx25b7 Mozmill Configs] | |||
=== Check List === | |||
* {{done|}} Automation ([http://mozmill-release.blargon7.com/#/functional/reports?branch=25.0&platform=All&from=2013-10-10&to=2013-10-11 report]) | |||
* {{done|}} Regression Testing ([https://moztrap.mozilla.org/results/cases/?filter-run=2411 report]) | |||
* {{done|}} Updates on ''betatest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=betatest&from=2013-10-10&to=2013-10-11&target=25.0 report]) | |||
* {{done|}} Updates on ''releasetest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=releasetest&from=2013-10-10&to=2013-10-11&target=25.0 report]) | |||
* {{done|}} Updates on ''beta'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=beta&from=2013-10-10&to=2013-10-11&target=25.0 report]) | |||
=== Regression Testing === | |||
The following areas have been identified for regression testing and will be covered via Moztrap smoketests. | |||
* '''Focus''': Breakpad, Bookmarks Library, Mixed Content Blocker, Add-on Compatibility | |||
* '''Platforms''': Windows 7 64-bit, Windows 8 32-bit, Windows 8.1 64-bit, Mac OSX 10.9, Ubuntu 12.04 32-bit | |||
* '''Instructions''': [https://moztrap.mozilla.org/runtests/environment/2411/ Run these tests] and [http://mzl.la/1eM3osi report bugs here]; add any issues found to the [[#Issues_Found_in_Testing|Issues Found]] section below | |||
== Beta 8 == | |||
=== Build Info === | |||
* Builds: [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0b8-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0b8-candidates/build1/ HTTP] | |||
* Build ID: 20131015052812 | |||
* [https://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=8b2c180a7b5d&tochange=f0c589ed7642 Change Log] | |||
* [https://wiki.mozilla.org/QA/Desktop_Firefox/Releases/Configs/Fx25b8 Mozmill Configs] | |||
=== Check List === | |||
* {{done|}} Automation ([http://mozmill-release.blargon7.com/#/functional/reports?branch=25.0&platform=All&from=2013-10-14&to=2013-10-15 report]) | |||
* {{done|}} Regression Testing ([https://moztrap.mozilla.org/results/cases/?filter-run=2453 report]) | |||
* {{done|}} Updates on ''betatest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=betatest&from=2013-10-15&to=2013-10-16&target=25.0 report]) | |||
* {{done|}} Updates on ''releasetest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=releasetest&from=2013-10-15&to=2013-10-16&target=25.0 report]) | |||
* {{done|}} Updates on ''beta'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=beta&from=2013-10-15&to=2013-10-16&target=25.0 report]) | |||
=== Regression Testing === | |||
The following areas have been identified for regression testing and will be covered via Moztrap smoketests. | |||
* '''Focus''': Breakpad, Software update for root-owned files, Downloads (multiple files - small and large ones, cancel, retry, open folder, open downloaded executables), PDF Viewer (open multiple pdf files and interact with them), Session Restore after restarting from regular and safe modes, Windows 8.1 exploratory testing, Mac OSX 10.9 exploratory testing | |||
* '''Platforms''': Windows XP, Windows 8.1 64-bit, Mac OSX 10.9, Ubuntu 13.10 32-bit | |||
* '''Instructions''': [https://moztrap.mozilla.org/runtests/environment/2453/ Run these tests] and [http://mzl.la/1eM3osi report bugs here]; add any issues found to the [[#Issues_Found_in_Testing|Issues Found]] section below | |||
== Beta 9 == | |||
=== Build Info === | |||
* Builds: [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0b9-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0b9-candidates/build1/ HTTP] | |||
* Build ID: 20131017174213 | |||
* [https://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=f0c589ed7642&tochange=18d6461ab468 Change Log] | |||
* [https://wiki.mozilla.org/QA/Desktop_Firefox/Releases/Configs/Fx25b9 Mozmill Configs] | |||
=== Check List === | |||
* {{done|}} Automation ([http://mozmill-release.blargon7.com/#/functional/reports?branch=25.0&platform=All&from=2013-10-17&to=2013-10-18 report]) | |||
* {{done|}} Regression Testing ([https://moztrap.mozilla.org/results/cases/?filter-run=2475 report]) | |||
* {{done|}} Updates on ''betatest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=betatest&from=2013-10-17&to=2013-10-18&target=25.0 report]) | |||
* {{done|}} Updates on ''releasetest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=releasetest&from=2013-10-17&to=2013-10-18&target=25.0 report]) | |||
* {{done|}} Updates on ''beta'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=beta&from=2013-10-17&to=2013-10-18&target=25.0 report]) | |||
=== Regression Testing === | |||
The following areas have been identified for regression testing and will be covered via Moztrap smoketests. | |||
* '''Focus''': Breakpad, Plug-in compatibility, Session Restore with multiple windows and tabs (exploratory testing with different options). | |||
* '''Platforms''': Windows XP, Windows 7 32-bit, Mac OSX 10.6, Ubuntu 12.04 64-bit | |||
* '''Instructions''': [https://moztrap.mozilla.org/runtests/environment/2475/ Run these tests] and [http://mzl.la/1eM3osi report bugs here]; add any issues found to the [[#Issues_Found_in_Testing|Issues Found]] section below | |||
== Beta 10 == | |||
=== Build Info === | |||
* Builds: [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0b10-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0b10-candidates/build1/ HTTP] | |||
* Build ID: 20131021191948 | |||
* [https://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=18d6461ab468&tochange=6393403b2015 Change Log] | |||
* [https://wiki.mozilla.org/QA/Desktop_Firefox/Releases/Configs/Fx25b10 Mozmill Configs] | |||
=== Check List === | |||
* {{done|}} Automation ([http://mozmill-release.blargon7.com/#/functional/reports?branch=25.0&platform=All&from=2013-10-22&to=2013-10-22 report]) | |||
* {{done|}} Regression Testing ([https://moztrap.mozilla.org/results/cases/?filter-run=2493 report]) | |||
* {{done|}} Updates on ''betatest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=betatest&from=2013-10-22&to=2013-10-22&target=25.0 report]) | |||
* {{done|}} Updates on ''releasetest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=releasetest&from=2013-10-22&to=2013-10-22&target=25.0 report]) | |||
* {{done|}} Updates on ''beta'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=beta&from=2013-10-22&to=2013-10-22&target=25.0 report]) | |||
=== Regression Testing === | |||
The following areas have been identified for regression testing and will be covered via Moztrap smoketests. | |||
* '''Focus''': Breakpad, Web Compatibility, PDF Viewer (open multiple pdf files and interact with them), Exit Firefox (close Firefox all possible ways when open with differently loaded profiles - many tabs, plugin content etc) | |||
* '''Platforms''': Windows XP, Windows 8.1 64-bit, Mac OSX 10.7, Ubuntu 13.10 64-bit | |||
* '''Instructions''': [https://moztrap.mozilla.org/runtests/environment/2493/ Run these tests] and [http://mzl.la/1eM3osi report bugs here]; add any issues found to the [[#Issues_Found_in_Testing|Issues Found]] section below | |||
== Beta 11 == | |||
=== Build Info === | |||
* Builds: [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0b11-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0b11-candidates/build1/ HTTP] | |||
* Build ID: 20131022211721 | |||
* [https://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=6393403b2015&tochange=e55593a833c9 Change Log] | |||
* [https://wiki.mozilla.org/QA/Desktop_Firefox/Releases/Configs/Fx25b11 Mozmill Configs] | |||
=== Check List === | |||
* {{done|}} Automation ([http://mozmill-release.blargon7.com/#/functional/reports?branch=25.0&platform=All&from=2013-10-23&to=2013-10-23 report]) | |||
* {{done|}} Regression Testing (see below) | |||
* {{done|}} Updates on ''betatest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=betatest&from=2013-10-23&to=2013-10-23&target=25.0 report]) | |||
* {{done|}} Updates on ''releasetest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=releasetest&from=2013-10-23&to=2013-10-23&target=25.0 report]) | |||
* {{done|}} Updates on ''beta'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=beta&from=2013-10-23&to=2013-10-23&target=25.0 report]) | |||
=== Regression Testing === | |||
The following areas have been identified for regression testing and will be covered via Moztrap smoketests. | |||
; OCSP Disabled | |||
* [https://moztrap.mozilla.org/manage/case/10557/ Use this Moztrap test] as a guide | |||
* Verify OCSP stapling is disabled and look for rendering, layout, and functional issues | |||
* Check as many SSL enabled websites as possible, listing any issues found in the section below | |||
* Note: the list below is a bare minimum, please add to it as time allows | |||
{| class="querytable" | |||
| class="header" | Sites Tested | |||
| class="header" | Windows XP | |||
| class="header" | Windows 7 32-bit | |||
| class="header" | Windows 8 64-bit | |||
| class="header" | Mac OSX 10.8 | |||
| class="header" | Ubuntu 13.04 32-bit | |||
|- | |||
| Amazon | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
|- | |||
| Baidu | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
|- | |||
| Bit.ly | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
|- | |- | ||
| | | Bugzilla | ||
| | | {{done|}} | ||
| | | {{done|}} | ||
| | | {{done|}} | ||
| {{ | | {{done|}} | ||
| {{done|}} | |||
|- | |||
| Digg | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
|- | |||
| EBay | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
|- | |||
| Facebook | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
|- | |||
| Flickr | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
|- | |||
| Google | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
|- | |||
| LinkedIn | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
|- | |||
| Outlook | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
|- | |||
| Pinterest | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
|- | |||
| QQ.com | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
|- | |||
| Reddit | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
|- | |||
| Slashdot | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
|- | |||
| SoundCloud | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
|- | |||
| Tumblr | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
| {{done|}} | |||
|- | |- | ||
| | | Twitter | ||
| | | {{done|}} | ||
| | | {{done|}} | ||
| | | {{done|}} | ||
| {{done|}} | | {{done|}} | ||
| {{done|}} | |||
|- | |- | ||
| | | Vimeo | ||
| | | {{done|}} | ||
| | | {{done|}} | ||
| | | {{done|}} | ||
| {{ | | {{done|}} | ||
| {{done|}} | |||
|- | |- | ||
| | | Wikipedia | ||
| | | {{done|}} | ||
| | | {{done|}} | ||
| | | {{done|}} | ||
| {{done|}} | | {{done|}} | ||
| {{done|}} | |||
|- | |- | ||
| | | Wordpress | ||
| | | {{done|}} | ||
| | | {{done|}} | ||
| | | {{done|}} | ||
| {{ | | {{done|}} | ||
| {{done|}} | |||
|- | |- | ||
| | | Yahoo | ||
| | | {{done|}} | ||
| | | {{done|}} | ||
| | | {{done|}} | ||
| {{done|}} | | {{done|}} | ||
| {{done|}} | |||
|- | |- | ||
| | | Yandex | ||
| | | {{done|}} | ||
| | | {{done|}} | ||
| | | {{done|}} | ||
| {{ | | {{done|}} | ||
| {{done|}} | |||
|- | |- | ||
| | | Youtube | ||
| | | {{done|}} | ||
| | | {{done|}} | ||
| | | {{done|}} | ||
| {{ | | {{done|}} | ||
| {{done|}} | |||
|- | |- | ||
|} | |} | ||
== | Issues Found: | ||
* No issues found | |||
* [http:// | |||
* [ | === Fix Verifications === | ||
* {{done|}} {{bug|929068}} Tumblr logo does not show correctly at first load. (style sheet would not load at first load) | |||
* {{done|}} {{bug|929524}} Layout is broken on twitter.com at initial loading | |||
== Beta 12 == | |||
=== Build Info === | |||
* Builds: [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0b12-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0b12-candidates/build1/ HTTP] | |||
* Build ID: 20131025150754 | |||
* [https://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=e55593a833c9&tochange=8c53a2c86cf4 Change Log] | |||
* [https://wiki.mozilla.org/QA/Desktop_Firefox/Releases/Configs/Fx25b12 Mozmill Configs] | |||
=== Check List === | |||
* | * {{done|}} Automation ([http://mozmill-release.blargon7.com/#/functional/reports?branch=25.0&platform=All&from=2013-10-26&to=2013-10-26 report]) | ||
* {{done|}} Updates on ''betatest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=betatest&from=2013-10-26&to=2013-10-26&target=25.0 report]) | |||
* | * {{done|}} Updates on ''releasetest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=releasetest&from=2013-10-26&to=2013-10-26&target=25.0 report]) | ||
* | * {{done|}} Updates on ''beta'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=beta&from=2013-10-26&to=2013-10-26&target=25.0 report]) | ||
=== Fix Verifications === | |||
* '' | * {{done|}} {{bug|927901}} Test has failed: Permission denied to access property 'encrypt' | ||
* {{done|}} {{bug|930827}} Hang when visiting icu-project.org | |||
== | == Release Candidate == | ||
=== Build Info === | |||
* | * Builds: <strike>[ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0-candidates/build1/ build1]</strike> <strike>[ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0-candidates/build2/ build2]</strike> [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0-candidates/build3/ build3] | ||
* | * Build ID: <strike>20131021192059</strike> <strike>20131022215409</strike> 20131025151332 | ||
* [https:// | * Change Log: <strike>[https://hg.mozilla.org/releases/mozilla-release/pushloghtml?fromchange=6393403b2015&tochange=8ce0230c65d2 build1]</strike> <strike>[https://hg.mozilla.org/releases/mozilla-release/pushloghtml?fromchange=8ce0230c65d2&tochange=de08ee190ca3 build2]</strike> [https://hg.mozilla.org/releases/mozilla-release/pushloghtml?fromchange=de08ee190ca3&tochange=d86ad7db1de3 build3] | ||
* [https://wiki.mozilla.org/QA/Desktop_Firefox/Releases/Configs/Fx25RC Mozmill Configs] | |||
=== Check List === | |||
* | * {{done|}} Automation ([http://mozmill-release.blargon7.com/#/functional/reports?branch=25.0&platform=All&from=2013-10-26&to=2013-10-29 report]) | ||
* | * {{done|}} Regression Testing (see below) | ||
* | * {{done|}} Updates on ''betatest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=betatest&from=2013-10-28&to=2013-10-28&target=25.0 report]) | ||
* | * {{done|}} Updates on ''releasetest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=releasetest&from=2013-10-28&to=2013-10-28&target=25.0 report]) | ||
* | * {{done|}} Updates on ''release'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=release&from=2013-10-29&to=2013-10-29&target=25.0 report]) | ||
* {{done|}} Verify update throttling level on release | |||
* {{done|}} Verify what's new page for 25.0-pl via {{bug|927975}} | |||
=== Regression Testing === | |||
The following areas have been identified for regression testing and will be covered via Moztrap smoketests. | |||
; Past Regressions | |||
* | * '''Focus''': Breakpad, NVidia 3D-Vision, Partner repack update, and Windows share compatibility smoketests | ||
* | * '''Platforms''': Windows XP, Windows 7 64-bit, Windows 8.1 32-bit, Mac OSX 10.9, Ubuntu 13.04 64-bit | ||
* | * '''Instructions''': [https://moztrap.mozilla.org/runtests/environment/2514/ Run these tests] and [http://mzl.la/1eM3osi report bugs here]; add any issues found to the [[#Issues_Found_in_Testing|Issues Found]] section below | ||
== | ; OCSP Disabled | ||
* [https://moztrap.mozilla.org/manage/case/10557/ Use this Moztrap test] as a guide | |||
| class="header" style= | * Verify OCSP stapling is disabled and look for rendering, layout, and functional issues | ||
| class="header" | * Check as many SSL enabled websites as possible, listing any issues found in the section below | ||
| class="header" | * Note: the list below is a bare minimum, please add to it as time allows | ||
{| class="querytable" | |||
| class="header" | Sites Tested | |||
| class="header" | Windows XP | |||
| class="header" | Windows 7 64-bit | |||
| class="header" | Windows 8.1 32-bit | |||
| class="header" | Mac OSX 10.9 | |||
| class="header" | Ubuntu 13.04 64-bit | |||
|- | |||
| [https://www.facebook.com/ Facebook] | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|- | |||
| [https://www.google.com Google] | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|- | |||
| [https://soundcloud.com/ SoundCloud] | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|- | |||
| [https://www.tumblr.com/ Tumblr] | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|- | |||
| [https://twitter.com Twitter] | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|- | |||
| [https://www.yahoo.com Yahoo] | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|- | |||
| [https://www.youtube.com/ Youtube] | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|- | |||
| [https://ib.btrl.ro/BT24 BT24] | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|- | |||
| [https://mail.google.com/ Google Mail] | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|- | |||
| [https://duckduckgo.com/ DuckDuckGo] | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|- | |||
| [https://login.live.com/ MSN] | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|- | |||
| [https://www.paypal.com/ PayPal] | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|- | |||
| [https://commerce.us.reuters.com/ Reuters] | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|- | |||
| [https://www.bbc.co.uk/news/ BBC News] | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|- | |||
| [https://moztrap.mozilla.org Moztrap] | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|- | |||
| [https://marketplace.firefox.com/ Marketplace] | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|- | |||
| [https://wiki.mozilla.org/ Mozilla Wiki] | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|- | |||
| [http://www.booking.com Booking.com] | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|- | |||
| [https://www.londonstockexchange.com London Stock Exchange] | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|- | |||
| [https://www.moneybookers.com/app/login.pl Moneybookers] | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|- | |||
| [https://academy.oracle.com/ Oracle Academy] | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|- | |||
| [http://www.sportsdirect.com/ Sports Direct] | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|- | |||
| [http://www.emag.ro/ Emag] | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|- | |||
| [http://www.imdb.com/ Imdb] | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|- | |||
| [http://www.aol.com/ Aol] | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|{{done|}} | |||
|- | |||
|} | |||
Issues Found: | |||
* No issues found | |||
=== Fix Verifications === | |||
* {{done|}} {{bug|929068}} Tumblr logo does not show correctly at first load. (style sheet would not load at first load) | |||
* {{done|}} {{bug|929524}} Layout is broken on twitter.com at initial loading | |||
* {{done|}} {{bug|927901}} Test has failed: Permission denied to access property 'encrypt' | |||
* {{done|}} {{bug|930827}} Hang when visiting icu-project.org | |||
* {{bug|927975}} Enable Firefox /whatsnew for /pl (polish) locale on Firefox 25 -- ''once updates are released'' | |||
== 25.0.1 == | |||
=== Build Info === | |||
* Builds: [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/25.0.1-candidates/build1/ build1] | |||
* Build ID: 20131112160018 | |||
* Change Log: [https://hg.mozilla.org/releases/mozilla-release/pushloghtml?fromchange=d86ad7db1de3&tochange=9fa84d1d4a61 build1] | |||
* [https://wiki.mozilla.org/QA/Desktop_Firefox/Releases/Configs/Fx25.0.1 Mozmill Configs] | |||
=== Check List === | |||
* {{done|}} Automation ([http://mozmill-release.blargon7.com/#/functional/reports?branch=25.0&platform=All&from=2013-11-13&to=2013-11-13 report]) | |||
* {{done|}} Regression Testing (see below) | |||
* {{done|}} Updates on ''betatest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=betatest&from=2013-11-13&to=2013-11-13&target=25.0.1 report]) | |||
* {{done|}} Updates on ''releasetest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=releasetest&from=2013-11-14&to=2013-11-15&target=25.0.1 report]) | |||
* {{done|}} Updates on ''release'' channel ([http://mozmill-release.blargon7.com/#/update/detail?branch=25.0&channel=release&from=2013-11-15&to=2013-11-16&target=25.0.1 report]) | |||
** This was done via manual spot checks due to problems with the Mozmill infrastructure. | |||
* {{done|}} Verify what's new page for 25.0.1-pl via {{bug|927975}} | |||
=== Regression Testing === | |||
The following areas have been identified for regression testing and will be covered via Moztrap smoketests. | |||
; Past Regressions | |||
* '''Focus''': Breakpad | |||
* '''Platforms''': Windows XP, Windows 7 64-bit, Windows 8.1 32-bit, Mac OSX 10.9, Ubuntu 13.04 64-bit | |||
* '''Instructions''': Force a crash with the [http://code.google.com/p/crashme/ crashme extension] and verify that the report is successfully submitted | |||
; SSL Topsite Testing | |||
* Navigate to a website which uses SSL and do some exploratory testing on that site | |||
* Be sure to log in to the website if it has that functionality | |||
* Report any issues with rendering and general usage, in particular if you get any SSL errors | |||
* Suggestions for sites to test include email, banking, social networking, shopping, video streaming, and forum websites | |||
* Try to test at least 25 different websites | |||
* Add ''PASS'' if there were no problems or ''FAIL'' if you encountered an issue | |||
* Be sure to note any issues in the ''Issues'' Found section below | |||
{| class="querytable" | |||
| class="header" | Sites Tested | |||
| class="header" | Windows XP | |||
| class="header" | Windows 7 64-bit | |||
| class="header" | Windows 8.1 32-bit | |||
| class="header" | Mac OSX 10.9 | |||
| class="header" | Ubuntu 13.04 64-bit | |||
|- | |||
| 1. https://www.facebook.com/ | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| 2. http://www.youtube.com/ | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| 3. https://login.yahoo.com/ | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| 4. https://www.linkedin.com/ | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| 5. https://login.live.com/ | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| 6. https://twitter.com/ | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| 7. http://www.amazon.com/ | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| 8. https://accounts.google.com/ | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| 9. http://wordpress.com/ | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| 10. http://www.ebay.com/ | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| 11. https://www.tumblr.com/ | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| 12. https://www.pinterest.com/ | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| 13. https://www.paypal.com/ | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| 14. https://instagram.com/accounts/login/ | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| 15. https://login.alibaba.com/ | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| 16. https://www.adcash.com/en/index.php | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| 17. https://vimeo.com/ | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| 18. http://www.aliexpress.com/ | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |- | ||
| | | 19. https://ib.btrl.ro/BT24/ | ||
| | | PASS | ||
| | | PASS | ||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |- | ||
| | | 20. https://www.pcgarage.ro | ||
| | | PASS | ||
| | | PASS | ||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |- | ||
| | | 21. https://www.emag.ro/ | ||
| | | PASS | ||
| | | PASS | ||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |- | ||
| | | 22. https://en.wikipedia.org/wiki/Main_Page | ||
| | | PASS | ||
| | | PASS | ||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| 23. https://soundcloud.com/ | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| 24. http://www.booking.com/ | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| 25. http://www.bbc.com | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| 26. http://eu.battle.net/en/ | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| 27. https://mail.google.com/ | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| 28. https://plus.google.com/ | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| 29. https://quality.mozilla.org | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| 30. http://www.foxnews.com | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
| PASS | |||
|- | |||
| '''add more if time permits''' | |||
| | |||
| | |||
| | |||
| | |||
| | |||
|- | |- | ||
|} | |} | ||
== | |||
Issues Found: | |||
* No issues found | |||
=== Fix Verifications === | |||
* {{done|}} {{bug|933733}} Page doesn't load/render if the mouse is not moving -- verified by reporter | |||
* {{done|}} {{bug|935959}} Update Mozilla to NSS 3.15.3 (new alternative NSS branch) to pick up a few fixes | |||
=== Bugs Reported === | |||
* {{ok|}} {{bug|938075}} Size of some Firefox 25.0.1 win32 l10n candidate builds is 45MB-72MB | |||
== Bugs == | |||
=== Potential Blockers === | |||
{{ok|}} | |||
* no tracked bugs | |||
{{done|}} | |||
* {{bug|921457}} [WebAudio] No sound is played when connecting a Delay Node | |||
* {{bug|904620}} [WebGL] Hangs in SurfaceStream_TripleBuffer_Async::WaitForCompositor | |||
* {{bug|929068}} Tumblr logo does not show correctly at first load. (style sheet would not load at first load) | |||
* {{bug|929617}} Validity period for stapled OCSP responses is too short | |||
* {{bug|929524}} Layout is broken on twitter.com at initial loading | |||
* {{bug|927975}} Enable Firefox /whatsnew for /pl (polish) locale on #Firefox25 | |||
{{drop|}} | |||
* {{bug|923319}} [WebAudio] Performance is extremely poor | |||
* {{bug|923106}} [WebAudio] Sound is not changed in the Oscillator audio source node | |||
* {{bug|923058}} [WebAudio] Jerky sound on Live Input node | |||
=== Issues Found in Testing === | |||
Add any new issues found or old issues reopened to the list below. These issues will be triaged and escalated by the release leads. | |||
<bugzilla> | <bugzilla> | ||
{ | { | ||
" | "id":"927421,925680,923058,913407,917818,912975,791195,788423,919938,920044,860249,920151,914997,862687,880854,870705,868905,904620,921457,921442,921449,921462,921469,921735,923061,856900,923106,78414, 913854, 925735, 928379, 928335, 929442", | ||
"include_fields":"id,summary,status" | |||
} | } | ||
</bugzilla> | </bugzilla> | ||
== | === Top Crashes === | ||
Conduct daily and weekly [https://etherpad.mozilla.org/stability-checks stability checks] and list any tracked bugs below. Any crash in the top-20 should have the ''topcrash'' keyword added and be nominated to track. Use [http://sisyphus.bughunter.ateam.phx1.mozilla.com/bughunter/login/ bughunter] to help with URL correlations. | |||
* {{done|}} | {{ok|}} | ||
* {{ | * no tracked bugs | ||
* {{ | |||
* {{ | {{done|}} | ||
* {{bug|929009}} crash in `anonymous namespace''::downmix_to_stereo<float>(float*, long, float*, int) | |||
* {{bug|925459}} crashes related to bitguard.dll | |||
* {{bug|921323}} crash in OpenGL@0x32af (MakeCurrent from OMTC) | |||
* {{bug|920725}} crash in nsHtml5StreamParser::WriteStreamBytes | |||
* {{bug|898156}} crash in mozilla::net::FTPChannelParent::OnStartRequest | |||
* {{bug|881636}} crash in mozilla::a11y::DocAccessible::UpdateTree | |||
* {{bug|873733}} crash in npjp2 @ mozilla::plugins::parent::_retainobject with Java SE7 | |||
* {{bug|800347}} crash in mozilla::ipc::RPCChannel::OnMaybeDequeueOne | |||
{{risk|}} | |||
* no bugs at risk | |||
{{drop|}} | |||
* {{bug|928020}} crash in mozalloc_abort(char const* const) | xul.dll@... | js::Invoke(JSContext*, JS::Value const&, JS::Value const&, unsigned int, JS::Value*, JS::MutableHandle<JS::Value>) | |||
* {{bug|927081}} crash in js::gc::IsAboutToBeFinalized<js::Shape> | |||
* {{bug|925407}} crash in npswf32_11_9_900_117.dll@0x345b8d | |||
* {{bug|812683}} [Win7] startup crash in _d2d_clear_surface @ CContext::RestorePipelineStateImpl (d3d11.dll, KB2670838, Asus Gamer OSD) | |||
* {{bug|767343}} crash in nsSupportsStringImpl::SetData with abort | |||
==== | === QA Wanted === | ||
{{ok|}} | |||
* no bugs ([http://mzl.la/1eah5Rq list]) | |||
* | |||
{{done|}} | |||
* {{ | * {{bug|929063}} Twitter layout broken/missing items | ||
* {{ | * {{bug|918612}} [Mac] Firefox 24 doesn't start when home directory is on AFP network share | ||
* {{ | * {{bug|913407}} Rendering issue in Addon manager | ||
* {{ | * {{bug|913043}} [adbe 3627468] Flash 11.9 crash in F1987069282 | ||
* {{bug|922875}} Stencil test fails even if there is no stencil buffer in the current FBO | |||
{{drop|}} | |||
* {{bug|919076}} Saved files permissions set incorrectly on Linux and Mac | |||
* | * {{bug|917011}} Video on second screen gets displayed on first screen in fullscreen mode | ||
* {{bug|886670}} Crash in mozilla::gl::SharedSurface_GLTexture::Fence @ libGPUSupportMercury on citadel demo guided tour | |||
* {{bug|698296}} crash in js::SweepBackgroundThings @ js::gc::Arena::finalize | |||
=== | === Verifications === | ||
* | ; How to Verify a Fix | ||
* [http://mzl.la/15ePeK2 Pick a bug from this list] | |||
* Download a build that is known to reproduce the bug and test to make sure you can see the bug | |||
* Download a build that is said to be fixed and test to make sure you cannot see the bug | |||
* If the bug reproduces in the fixed build: | |||
** set the STATUS field to REOPENED and provide a comment detailing your testing | |||
* If the bug does not reproduce in the fixed build: | |||
** set the status flag to ''verified'' for the version you tested | |||
** set the STATUS field to VERIFIED FIXED if the version tested is the same as the TARGET MILESTONE | |||
** any bugs marked ''verified'' will automatically appear in the ''Verified Fixes'' section below | |||
; Verified Fixes | |||
<bugzilla> | |||
{ | |||
"include_fields":"id, summary", | |||
"f1":"op_sys", | |||
"o1":"nowordssubstr", | |||
"v1":"Android Gonk", | |||
"f2":"keywords", | |||
"o2":"nowordssubstr", | |||
"v2":"assertion intermittent verifyme", | |||
"f3":"component", | |||
"o3":"nowordssubstr", | |||
"v3":"Build%20Config", | |||
"f4":"product", | |||
"o4":"nowordssubstr", | |||
"v4":"QA mozilla SDK Boot2Gecko Testing", | |||
"f5":"status_whiteboard", | |||
"o5":"nowordssubstr", | |||
"v5":"qa asan Leo", | |||
"cf_status_firefox25":"verified" | |||
} | |||
</bugzilla> | |||
=== | === Unconfirmed === | ||
[http://mzl.la/14zJWdJ Pick a bug from this list] and try to reproduce the issue using the information provided. If you reproduce the issue, provide a description of the circumstances, set the status to NEW, and move the bug into a component which matches the issue (ie. a bookmarks issue would go in the Bookmarks component). If you can't reproduce the issue ask the reporter for more information by setting the need-info? flag to their email address. | |||
; Confirmed Bugs | |||
<bugzilla> | |||
{ | |||
"bug_status":"NEW", | |||
"f1":"op_sys", | |||
"o1":"nowordssubstr", | |||
"v1":"Android Gonk", | |||
"f2":"keywords", | |||
"o2":"nowordssubstr", | |||
"v2":"assertion intermittent verifyme", | |||
"f3":"component", | |||
"o3":"nowordssubstr", | |||
"v3":"Build%20Config Tests", | |||
"f4":"product", | |||
"o4":"nowordssubstr", | |||
"v4":"QA mozilla SDK Boot2Gecko Testing Thunderbird", | |||
"f5":"status_whiteboard", | |||
"o5":"nowordssubstr", | |||
"v5":"qa asan Leo", | |||
"f6":"version", | |||
"o6":"anywordssubstr", | |||
"v6":"25" | |||
} | |||
</bugzilla> | |||
=== Other Bugs === | |||
The following other queries should be triaged periodically, mostly by the QA Driver, and nominated for testing. | |||
* [http://mzl.la/13CTou8 regressionwindow-wanted]: nominate for qawanted if testing is needed or remove regressionwindow-wanted if the request is no longer valid | |||
* [ fixed]: nominate for verifyme if QA testing is needed, [easy verify] of community testing is needed, or [qa-] if no testing is needed | |||
==== | == New Platform Readiness == | ||
=== Windows 8.1 === | |||
{| class="querytable" | |||
|- | |||
| style="width:15%" | '''Status''' | |||
| <span style="color:green">'''COMPLETE'''</span> (due 16 October 2013) | |||
|- | |||
| '''Owner''' | |||
| Anthony Hughes | |||
|- | |||
| '''Manual Testing''' | |||
| <span style="color:green">'''COMPLETE'''</span> (see known bugs listed below) | |||
|- | |||
| '''Automated Testing''' | |||
| <span style="color:green">'''COMPLETE'''</span> (see known bugs listed below) | |||
|- | |||
| '''Bugs''' | |||
| | |||
{{ok|}} | |||
* {{bug|922314}} Add MFT Support for WebM | |||
* {{bug|924045}} Metro icon shown after fresh install even if nightly is not the default browser | |||
{{done|}} | |||
* {{bug|921080}} Clicking on item in download panel not working on Windows 8.1 RTM | |||
* {{bug|924038}} Snapped view in Windows 8.1 64-bit | |||
* {{bug|924051}} Display rendering issue if 2 monitors are used | |||
* {{bug|923241}} Crash @ RtlAcquireSRWLockExclusive in d3d11.dll@0x17214 | |||
|} | |||
=== | === Mac OSX 10.9 === | ||
* | {| class="querytable" | ||
* | |- | ||
* | | style="width:15%" | '''Status''' | ||
| <span style="color:green">'''COMPLETE'''</span> (due TBD October 2013) | |||
|- | |||
| '''Owner''' | |||
| Marcia Knous | |||
|- | |||
| '''Manual Testing''' | |||
| <span style="color:green">'''COMPLETE'''</span> (see known bugs listed below) | |||
|- | |||
| '''Automated Testing''' | |||
| <span style="color:green">'''COMPLETE'''</span> (integrated into Mozmill-CI workflow) | |||
|- | |||
| '''Bugs''' | |||
| | |||
{{ok|}} | |||
* ''{{bug|917042}} File picker often closes when trying to select a file the first time'' -- Needs regression range | |||
* ''{{bug|888605}} Crash in [NSPopUpButtonObserver menuChangedItem]'' -- Needs regression range | |||
* ''{{bug|883859}} Dialogs do not open on the screen where Firefox is active'' -- Possible Tech Evangelism | |||
{{done|}} | |||
* {{bug|918943}} Duplicate symbol errors linking WebRTC when using the 10.9 SDK with --disable-optimize | |||
* {{bug|917526}} Various duplicate symbols error building tree with the 10.9 SDK | |||
* {{bug|901348}} Duplicate symbol errors building --with-intl-api | |||
* {{bug|886096}} Crash in libdispatch'' | |||
|} | |||
== | == Post-mortem == | ||
* [https://mozqa.etherpad.mozilla.org/firefox25-post-mortem Agenda & Notes] | |||
* Follow-up: | |||
** OCSP - not tested well enough due to lack of live sites to test against | |||
*** QA needs to be able to test features even without live use cases | |||
*** Should Mozilla try to find a "launch partner" for these types of features in the future? | |||
** WebCrypto - undocumented uplift to Firefox 25 | |||
*** QA needs to be "involved" in uplift approvals, attending the RelMan meetings is one way we can achieve this | |||
** App Manager - flew under the radar completely | |||
*** QA needs to escalate these "failures" as soon as they happen | |||
*** Talk to ''dkl'' or ''glob'' about identifying "features" based on Bugzilla heuristics (patch size, number of files, component, etc) | |||
** l10n (24esr) - RelMan's mistake | |||
*** QA is incapable of identifying these issues via manual testing and automation | |||
*** RelEng should harden their system to mitigate exposure to human error | |||
*** QA should try to involve l10n more in community events, reach out to localizers more | |||
** Mozmill-CI Instability | |||
*** Cannot continue to force European staff to troubleshoot when they are on PTO, sick, or should be sleeping | |||
*** A-Team needs someone in a North American timezone to maintain and troubleshoot issues during Beta/Release pushes |
Latest revision as of 23:06, 15 November 2013
« Firefox 24 | Firefox 25 | Firefox 26 »
Firefox 25 Test Plan
Summary
The following is the test plan for Firefox 25 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 25. After the release this document will be used as an archive of what was done to validate this release.
Status | Currently in Aurora -- merging to Beta on September 19, 2013 |
Lead | Anthony Hughes (irc: ashughes) |
Builds | aurora.mozilla.org |
- If you have some free time, please pick a task below and get in touch with one of the leads.
Schedule
June 24, 2013 | Firefox 25 lands on Nightly |
August 5, 2013 | Firefox 25 moves up to Aurora |
September 19, 2013 | Firefox 25 moves up to Beta |
October 22, 2013 | Firefox 25 Release Candidate available for testing |
October 29, 2013 | Firefox 25 is released |
Aurora
Branch Migration
Pre Sign-off Tasks
- [DONE] Firefox branch migrated (Release Management)
- [DONE] Mozmill Tests branch migrated (Automation Development)
- [DONE] Updates throttled on aurora channel (Release Engineering)
- [DONE] Updates verified throttled on aurora channel (QA)
Sign-off Tasks
- [DONE] Mozmill Daily functional testruns pass with no unexpected failures/regressions (report)
- [DONE] Mozmill Daily l10n testruns pass with no unexpected failures/regressions (report)
- [DONE] Mozmill Daily endurance testruns pass with no unexpected failures/regressions (report)
- [DONE] Mozmill Daily remote testruns pass with no unexpected failures/regressions (report)
- [DONE] Mozmill Daily auroratest update testruns pass with no unexpected failures/regressions (report)
Post Sign-off Tasks
- [DONE] Aurora sign-off email sent (QA)
- [DONE] Updates unthrottled on aurora channel (Release Engineering)
- [DONE] Updates verified unthrottled on aurora channel (QA)
- [MISSED] Verify Stub Installers stage the new Aurora build (bug 910442)
Pre-Beta Sign-off
Checklist
- [DONE] Mozmill-CI automation testruns checked for potential blocking regressions
- [DONE] Features signed-off for Beta readiness, verified disabled if not
- [ON TRACK] WebRTC spotchecked for potential blocking regressions, parity with Firefox 22
- [ON TRACK] Flagged bug fixes verified
Automation
The following tracks the results of daily automation run against the Aurora builds as indicated by the Mozmill Addons, Endurance, Functional, L10n, Remote, and Update reports.
Top Failures | Platform | Instances | Bug | Status |
/testSecurity/testGreenLarry.js | All | 130 | bug 907101 | FIXED (2013-08-21) |
/testPreferences/testRestoreHomepageToDefault.js | All | 41 | bug 912363 | DISABLED (2013-09-05) |
/testPopups/testPopupsAllowed.js | All | 36 | bug 790218 | FIXED (2013-09-09) |
/testSecurity/testSafeBrowsingNotificationBar.js | All | 32 | bug 905033 | DISABLED (2013-08-12) |
/testFindInPage/testFindInPage.js | All | 24 | bug 909227 | BLOCKED by l10n bug 913396 (2013-09-05) |
/testSecurity/testDVCertificate.js | All | 24 | bug 890487 | FIXED (2013-08-06) |
/testSecurity/testMixedContentPage.js | All | 24 | bug 901951 | FIXED (2013-08-06) |
/testPrivateBrowsing/testAboutCache.js | Windows | 23 | bug 888924 | ASSIGNED (2013-09-05) |
/testPopups/testPopupsBlocked.js | Windows | 13 | n/a | WORKSFORME (2013-09-04) |
Resident Memory Usage | Week 1 | Week 2 | Week 3 | Week 4 | Week 5 | Week 6 | Net |
Windows XP | 105 MB | 106 MB | 106 MB | 106 MB | 106 MB | 104 MB | -0.952% |
Windows Vista | 97 MB | 97 MB | 97 MB | 97 MB | 97 MB | 97 MB | no change |
Windows 7 32-bit | 100 MB | 100 MB | 100 MB | 100 MB | 100 MB | 99 MB | -1.000% |
Windows 7 64-bit | 102 MB | 102 MB | 102 MB | 102 MB | 102 MB | 101 MB | -0.980% |
Windows 8 32-bit | 130 MB | 124 MB | 122 MB | 123 MB | 122 MB | 102 MB | -21.538% |
Windows 8 64-bit | 132 MB | 126 MB | 123 MB | 124 MB | 124 MB | 104 MB | -21.212% |
Mac OSX 10.6 | 138 MB | 134 MB | 134 MB | 134 MB | 134 MB | 132 MB | -4.348% |
Mac OSX 10.7 | 203 MB | 200 MB | 200 MB | 201 MB | 199 MB | 197 MB | -2.956% |
Mac OSX 10.8 | 195 MB | 189 MB | 190 MB | 190 MB | 191 MB | 192 MB | -1.538% |
Ubuntu LTS 32-bit | 103 MB | 100 MB | 100 MB | 100 MB | 98 MB | 98 MB | -4.854% |
Ubuntu LTS 64-bit | 135 MB | 131 MB | 131 MB | 131 MB | 131 MB | 129 MB | -4.444% |
Ubuntu Desktop 32-bit | 102 MB | 100 MB | 100 MB | 100 MB | 101 MB | 99 MB | -2.941% |
Ubuntu Desktop 64-bit | 135 MB | 131 MB | 131 MB | 131 MB | 132 MB | 130 MB | -3.704% |
Features
The following features are tracked and tested as indicated by the release notes, relnote+, Firefox, and Platform feature bugs.
- [DONE] Web Audio (Manuela Muntean)
- [DONE] Find Bar Redesign (Manuela Muntean)
- [DONE] Reset Firefox Saves Windows and Tabs (Ioana Budnar)
- [DONE] Reset Profile When Unused for Months (Simona Marcu)
- [DONE] CSS3 background-attachment:local Support (Petruta Rasa)
- [DONE] Larry Panel (Bogdan Maris)
- [DONE] Emacs Map#forEach and Set#forEach (Paul Silaghi)
- [DONE] ES6 math functions (Mihai Morar)
- [DONE] Support in-line HTML5 iframes (Alexandra Lucinet)
- [DONE] Session Restore backend work, tab close & restore (Cornel Ionce)
- [DROPPED] Background Thumbnail Service (Paul Silaghi)
- [DROPPED] Prototype DOM Gamepad API (Alexandra Lucinet)
WebRTC
Using the Moztrap test as a guideline, verify that Firefox 25.0a2 is no worse than Firefox 22.0 when making AppRTC calls of >5 minutes in length. When testing, here are the guidelines to follow:
- Have only one browser open on each machine at any given time
- Have only one call running at any given time on a particular machine
- Make sure the caller and callee are always on *different* machines.
- Please don't test any 3+-way calls for this sanity check -- We just want to see the results for 1:1 (basic) calling
- If you find regressions, report a bug and CC Randell Jesup and Maire Reavy; they can help track down if the regressions are real or not
- Tip: be sure to provide extremely detailed steps to reproduce and witnessed results, as well as detailed information about your test environment; more information is better than not enough information.
Caller | Callee | Result |
Firefox 25.0a2 on Windows 7 | Firefox 25.0a2 on Windows 7 | PASS |
Firefox 22.0 on Windows 7 | Firefox 25.0a2 on Windows XP | FAIL (1*) |
Chrome Stable on Windows 7 | Firefox 25.0a2 on Windows 8 | PASS |
Firefox 25.0a2 on Windows 7 | Firefox 22.0 on Mac OSX 10.8 | PASS (2*) |
Firefox 25.0a2 on Windows 7 | Chrome Stable on Ubuntu 12.04 | PASS |
Firefox 22.0 on Windows XP | Firefox 25.0a2 on Windows 7 | FAIL (4*) |
Firefox 25.0a2 on Windows XP | Firefox 22.0 on Windows XP | FAIL (10*) |
Firefox 25.0a2 on Windows XP | Chrome Stable on Windows 8 | FAIL (5*) |
Chrome Stable on Windows XP | Firefox 25.0a2 on Mac OSX 10.8 | PASS |
Firefox 22.0 on Windows XP | Firefox 25.0a2 on Ubuntu 13.04 | FAIL (10*) |
Chrome Stable on Windows 8 | Firefox 25.0a2 on Windows 7 | FAIL (8*) |
Firefox 25.0a2 on Windows 8 | Chrome Stable on Windows XP | FAIL (7*) |
Firefox 22.0 on Windows 8 | Firefox 25.0a2 on Windows 8 | PASS |
Firefox 25.0a2 on Windows 8 | Firefox 22.0 on Windows 8 | PASS |
Firefox 22.0 on Windows 8 | Firefox 25.0a2 on Mac OSX 10.8 | FAIL (6*) |
Chrome Stable on Windows 8 | Firefox 25.0a2 on Ubuntu 13.04 | PASS |
Firefox 25.0a2 on Mac OSX 10.8 | Firefox 22.0 on Windows 7 | FAIL (9*) |
Chrome Stable on Mac OSX 10.8 | Firefox 25.0a2 on Windows XP | FAIL (9*) |
Firefox 22.0 on Mac OSX 10.8 | Firefox 25.0a2 on Windows 8 | FAIL (11*) |
Firefox 25.0a2 on Mac OSX 10.8 | Chrome Stable on Mac OSX 10.8 | PASS |
Firefox 25.0a2 on Mac OSX 10.8 | Firefox 22.0 on Ubuntu 13.04 | FAIL (9*) |
Firefox 25.0a2 on Ubuntu 12.04 | Chrome Stable on Windows 7 | PASS |
Firefox 22.0 on Ubuntu 12.04 | Firefox 25.0a2 on Windows XP | PASS |
Chrome Stable on Ubuntu 12.04 | Firefox 25.0a2 on Windows 8 | PASS |
Firefox 25.0a2 on Ubuntu 12.04 | Firefox 22.0 on Mac OSX 10.8 | PASS |
Firefox 25.0a2 on Ubuntu 12.04 | Chrome Stable on Ubuntu 13.04 | PASS |
Detailed Results
- (1*) The caller hears himself, as well. The callee hears delayed sound (1-2sec) after a few minutes
- not reproducible if caller uses latest Aurora 25.0a2.
- (2*) The caller got a crash after ~12 min, while opening a page in a new tab: https://crash-stats.mozilla.com/report/index/97cf87a4-284f-4cd0-9a17-ed6cf2130906
- one time occurrence, not related to WebRTC
- (4*) Both caller and callee hear delayed sound (1-2secs) - reproducible also if callee uses Firefox 23.0.1
- (5*) The caller hears delayed sound
- reproducible also with FF23.0.1 and 24b9; moreover, with 23.0.1 and 24b.9, the delay appears on both sides
- (6*) The callee hear himself, as well
- reproducible also with 23.0.1 and Firefox 24 beta 10
- (7*) Callee's conversation page crashes when caller attends
- reproducible with FF23.0.1 or 24b9 (used by caller)
- (8*) when caller connects, both attendances remain stuck in “Connecting…” mode
- working with 23.0.1 and 24b9
- (9*) when caller connects, both attendances remain stuck in “Connecting…” mode
- working with 23.0.1 and 24b9
- the problem is the TURN server which is giving a bogus address
- (10*) sometimes, the image and sound is choppy and there is a delayed sound: ~5 seconds at caller, ~10seconds at callee; callee hears himself, as well
- reproducible with 23.0.1
- (11*) attendants see themselves instead of seeing the other person in the call and there is no sound
- reproducible with 23.0.1
Bug Fix Verifications
The following bugs have been fixed and flagged for verification. At a minimum, any critical/blocker bug should be verified before migration to Beta.
5 Total; 0 Open (0%); 2 Resolved (40%); 3 Verified (60%);
Potential Blockers
The following bugs have been nominated to block this milestone from uplifting to Beta.
- no blockers currently
Beta 1
Build Info
- Builds: FTP, HTTP
- Build ID: 20130917123208
- Change Log
- Mozmill Configs
Check List
Criteria | Description | Status |
Automation | No unexpected test failures from Mozmill functional testruns | SIGNED-OFF (report) |
Regression Testing | No unexpected failures from manually smoketesting past regressions and new features | SIGNED-OFF (report) |
Bug Fix Verifications | No unexpected failures from manually verifying landed bug fixes | SIGNED-OFF (report) |
Stability Verification | No unexpected spikes in top crash report for this milestone | SIGNED-OFF (report) |
Updates on betatest channel | No unexpected failures from Mozmill update testruns on the betatest channel | SIGNED-OFF (report) |
Updates on releasetest channel | No unexpected failures from Mozmill update testruns on the releasetest channel | SIGNED-OFF (report) |
Updates on betatest channel | No unexpected failures from Mozmill update testruns on the beta channel | SIGNED-OFF (report) |
Regression Testing
The following areas have been identified for regression testing and will be covered via Moztrap smoketests.
- Test Coverage
- Breakpad
- NVidia 3D-Vision
- WebRTC AppRTC 1:1 calling (subset of Aurora testing)
- DefaultKeyBinding.dict support on Mac OSX (Paul Silaghi)
- Web Audio (Manuela Muntean)
- Find Bar Redesign (Manuela Muntean)
- Reset Firefox Saves Windows and Tabs (Ioana Budnar)
- Reset Profile When Unused for Months (Simona Marcu)
- Larry Panel (Bogdan Maris)
- Session Restore backend work, tab close & restore (Cornel Ionce)
- Platform Coverage
- Windows XP
- Windows 7 64-bit
- Windows 8.1 Preview 64-bit
- Mac OSX 10.8
- Ubuntu 13.04 64-bit
- Instructions
- Run these tests
- Select one of the platforms
- Follow the steps in each of the tests and report your results within Moztrap
- When you get to the WebRTC smoketest refer to the Special Coverage for WebRTC section below for coverage details
- If you encounter a bug, report it and add it to the Issues list below using the bug number template
- Special Coverage for WebRTC
- Caller Firefox/Platform -> Callee Firefox/Platform
- Firefox 25.0b1 on Windows XP -> Firefox 25.0b1 on Windows 7
- Firefox 25.0b1 on Windows 7 -> Firefox 25.0b1 on Windows 8
- Firefox 25.0b1 on Windows 8 -> Firefox 25.0b1 on Mac OSX 10.8
- Firefox 25.0b1 on Mac OSX 10.8 -> Firefox 25.0b1 on Ubuntu 13.04
- Firefox 25.0b1 on Ubuntu 13.04 -> Firefox 25.0b1 on Windows XP
Issues
List any new issues found or old issues reopened using the bug number template. These issues will be triaged and escalated by the release leads.
Bug | Status |
bug 917818 - Findbar buttons remain in pressed state on Linux | NEW - does not block 25b1 |
bug 912975 - Windows and tabs are not always restored after Reseting Firefox | NEW - does not block 25b1 |
bug 860249 - [Mozmill] Exception "Sorry, cannot connect to jsbridge extension, port 24242" | NEW - does not block 25b1 |
bug 791195 - Session is not restored after Firefox crash | NEW - does not block 25b1 |
Beta 2
Build Info
- Builds: FTP, HTTP
- Build ID: 20130923194050
- Change Log
- Mozmill Configs
Check List
Criteria | Description | Status |
Automation | No unexpected test failures from Mozmill functional testruns | SIGNED-OFF (report) |
Regression Testing | No unexpected failures from manually smoketesting past regressions and new features | SIGNED-OFF (report) |
Bug Fix Verifications | No unexpected failures from manually verifying landed bug fixes | SIGNED-OFF (report) |
Stability Verification | No unexpected spikes in top crash report for this milestone | SIGNED-OFF (report) |
Updates on betatest channel | No unexpected failures from Mozmill update testruns on the betatest channel | SIGNED-OFF (report) |
Updates on releasetest channel | No unexpected failures from Mozmill update testruns on the releasetest channel | SIGNED-OFF (report) |
Updates on beta channel | No unexpected failures from Mozmill update testruns on the beta channel | SIGNED-OFF (report) |
Regression Testing
The following areas have been identified for regression testing and will be covered via Moztrap smoketests.
- Test Coverage
- Breakpad
- MD5 Hash De-support
- Plug-in Compatibility
- Exploratory testing around Flash 11.9 Beta : etherpad
- Platform Coverage
- Windows Vista
- Windows 7 32-bit
- Windows 8 64-bit
- Mac OSX 10.7
- Ubuntu 12.04 64-bit
- Instructions
- Run these tests
- Select one of the platforms
- Follow the steps in each of the tests and report your results within Moztrap
- If you encounter a bug, report it and add it to the Issues list below using the bug number template
Beta 3
Build Info
- Builds: FTP, HTTP
- Build ID: 20130926170421
- Change Log
- Mozmill Configs
Check List
- [DONE] Automation (report)
- [DONE] Regression Testing (report)
- [DONE] Updates on betatest channel (report)
- [DONE] Updates on releasetest channel (report)
- [DONE] Updates on beta channel (report)
Regression Testing
The following areas have been identified for regression testing and will be covered via Moztrap smoketests.
- Focus: Breakpad, Web Audio, WebGL
- Platforms: Windows XP, Windows 7 64-bit, Windows 8.1 32-bit, Mac OSX 10.8, Ubuntu 13.04 64-bit
- Instructions: Run these tests and report bugs here; add any issues found to the Issues Found section below
Beta 4
Build Info
- Builds: FTP, HTTP
- Build ID: 20131001024718
- Change Log
- Mozmill Configs
Check List
- [DONE] Automation (report)
- [DONE] Regression Testing (report)
- [DONE] Updates on betatest channel (report)
- [DONE] Updates on releasetest channel (report)
- [DONE] Updates on beta channel (report)
Regression Testing
The following areas have been identified for regression testing and will be covered via Moztrap smoketests.
- Focus: Breakpad, Web Audio, WebGL
- Platforms: Windows XP, Windows Vista, Windows 8 64-bit, Mac OSX 10.7, Ubuntu 12.04 64-bit
- Instructions: Run these tests and report bugs here; add any issues found to the Issues Found section below
Beta 6
Build Info
- Builds: FTP, HTTP
- Build ID: 20131007213254
- Change Log
- Mozmill Configs
Check List
- [DONE] Automation (report)
- [DONE] Regression Testing (report)
- [DONE] Updates on betatest channel (report)
- [DONE] Updates on releasetest channel (report)
- [DONE] Updates on beta channel (report)
Regression Testing
The following areas have been identified for regression testing and will be covered via Moztrap smoketests.
- Focus: Breakpad, Web Audio, WebGL, TLS/SSL
- Platforms: Windows XP, Windows Vista, Windows 8 64-bit, Mac OSX 10.7, Ubuntu 12.04 64-bit
- Instructions: Run these tests and report bugs here; add any issues found to the Issues Found section below
- Exploratory testing: TLS/SSL etherpad
Beta 7
Build Info
- Builds: FTP, HTTP
- Build ID: 20131010180222
- Change Log
- Mozmill Configs
Check List
- [DONE] Automation (report)
- [DONE] Regression Testing (report)
- [DONE] Updates on betatest channel (report)
- [DONE] Updates on releasetest channel (report)
- [DONE] Updates on beta channel (report)
Regression Testing
The following areas have been identified for regression testing and will be covered via Moztrap smoketests.
- Focus: Breakpad, Bookmarks Library, Mixed Content Blocker, Add-on Compatibility
- Platforms: Windows 7 64-bit, Windows 8 32-bit, Windows 8.1 64-bit, Mac OSX 10.9, Ubuntu 12.04 32-bit
- Instructions: Run these tests and report bugs here; add any issues found to the Issues Found section below
Beta 8
Build Info
- Builds: FTP, HTTP
- Build ID: 20131015052812
- Change Log
- Mozmill Configs
Check List
- [DONE] Automation (report)
- [DONE] Regression Testing (report)
- [DONE] Updates on betatest channel (report)
- [DONE] Updates on releasetest channel (report)
- [DONE] Updates on beta channel (report)
Regression Testing
The following areas have been identified for regression testing and will be covered via Moztrap smoketests.
- Focus: Breakpad, Software update for root-owned files, Downloads (multiple files - small and large ones, cancel, retry, open folder, open downloaded executables), PDF Viewer (open multiple pdf files and interact with them), Session Restore after restarting from regular and safe modes, Windows 8.1 exploratory testing, Mac OSX 10.9 exploratory testing
- Platforms: Windows XP, Windows 8.1 64-bit, Mac OSX 10.9, Ubuntu 13.10 32-bit
- Instructions: Run these tests and report bugs here; add any issues found to the Issues Found section below
Beta 9
Build Info
- Builds: FTP, HTTP
- Build ID: 20131017174213
- Change Log
- Mozmill Configs
Check List
- [DONE] Automation (report)
- [DONE] Regression Testing (report)
- [DONE] Updates on betatest channel (report)
- [DONE] Updates on releasetest channel (report)
- [DONE] Updates on beta channel (report)
Regression Testing
The following areas have been identified for regression testing and will be covered via Moztrap smoketests.
- Focus: Breakpad, Plug-in compatibility, Session Restore with multiple windows and tabs (exploratory testing with different options).
- Platforms: Windows XP, Windows 7 32-bit, Mac OSX 10.6, Ubuntu 12.04 64-bit
- Instructions: Run these tests and report bugs here; add any issues found to the Issues Found section below
Beta 10
Build Info
- Builds: FTP, HTTP
- Build ID: 20131021191948
- Change Log
- Mozmill Configs
Check List
- [DONE] Automation (report)
- [DONE] Regression Testing (report)
- [DONE] Updates on betatest channel (report)
- [DONE] Updates on releasetest channel (report)
- [DONE] Updates on beta channel (report)
Regression Testing
The following areas have been identified for regression testing and will be covered via Moztrap smoketests.
- Focus: Breakpad, Web Compatibility, PDF Viewer (open multiple pdf files and interact with them), Exit Firefox (close Firefox all possible ways when open with differently loaded profiles - many tabs, plugin content etc)
- Platforms: Windows XP, Windows 8.1 64-bit, Mac OSX 10.7, Ubuntu 13.10 64-bit
- Instructions: Run these tests and report bugs here; add any issues found to the Issues Found section below
Beta 11
Build Info
- Builds: FTP, HTTP
- Build ID: 20131022211721
- Change Log
- Mozmill Configs
Check List
- [DONE] Automation (report)
- [DONE] Regression Testing (see below)
- [DONE] Updates on betatest channel (report)
- [DONE] Updates on releasetest channel (report)
- [DONE] Updates on beta channel (report)
Regression Testing
The following areas have been identified for regression testing and will be covered via Moztrap smoketests.
- OCSP Disabled
- Use this Moztrap test as a guide
- Verify OCSP stapling is disabled and look for rendering, layout, and functional issues
- Check as many SSL enabled websites as possible, listing any issues found in the section below
- Note: the list below is a bare minimum, please add to it as time allows
Sites Tested | Windows XP | Windows 7 32-bit | Windows 8 64-bit | Mac OSX 10.8 | Ubuntu 13.04 32-bit |
Amazon | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Baidu | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Bit.ly | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Bugzilla | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Digg | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
EBay | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
[DONE] | [DONE] | [DONE] | [DONE] | [DONE] | |
Flickr | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
[DONE] | [DONE] | [DONE] | [DONE] | [DONE] | |
[DONE] | [DONE] | [DONE] | [DONE] | [DONE] | |
Outlook | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
[DONE] | [DONE] | [DONE] | [DONE] | [DONE] | |
QQ.com | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
[DONE] | [DONE] | [DONE] | [DONE] | [DONE] | |
Slashdot | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
SoundCloud | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Tumblr | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
[DONE] | [DONE] | [DONE] | [DONE] | [DONE] | |
Vimeo | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Wikipedia | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Wordpress | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Yahoo | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Yandex | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Youtube | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Issues Found:
- No issues found
Fix Verifications
- [DONE] bug 929068 Tumblr logo does not show correctly at first load. (style sheet would not load at first load)
- [DONE] bug 929524 Layout is broken on twitter.com at initial loading
Beta 12
Build Info
- Builds: FTP, HTTP
- Build ID: 20131025150754
- Change Log
- Mozmill Configs
Check List
- [DONE] Automation (report)
- [DONE] Updates on betatest channel (report)
- [DONE] Updates on releasetest channel (report)
- [DONE] Updates on beta channel (report)
Fix Verifications
- [DONE] bug 927901 Test has failed: Permission denied to access property 'encrypt'
- [DONE] bug 930827 Hang when visiting icu-project.org
Release Candidate
Build Info
- Builds:
build1build2build3 - Build ID:
201310211920592013102221540920131025151332 - Change Log:
build1build2build3 - Mozmill Configs
Check List
- [DONE] Automation (report)
- [DONE] Regression Testing (see below)
- [DONE] Updates on betatest channel (report)
- [DONE] Updates on releasetest channel (report)
- [DONE] Updates on release channel (report)
- [DONE] Verify update throttling level on release
- [DONE] Verify what's new page for 25.0-pl via bug 927975
Regression Testing
The following areas have been identified for regression testing and will be covered via Moztrap smoketests.
- Past Regressions
- Focus: Breakpad, NVidia 3D-Vision, Partner repack update, and Windows share compatibility smoketests
- Platforms: Windows XP, Windows 7 64-bit, Windows 8.1 32-bit, Mac OSX 10.9, Ubuntu 13.04 64-bit
- Instructions: Run these tests and report bugs here; add any issues found to the Issues Found section below
- OCSP Disabled
- Use this Moztrap test as a guide
- Verify OCSP stapling is disabled and look for rendering, layout, and functional issues
- Check as many SSL enabled websites as possible, listing any issues found in the section below
- Note: the list below is a bare minimum, please add to it as time allows
Sites Tested | Windows XP | Windows 7 64-bit | Windows 8.1 32-bit | Mac OSX 10.9 | Ubuntu 13.04 64-bit |
[DONE] | [DONE] | [DONE] | [DONE] | [DONE] | |
[DONE] | [DONE] | [DONE] | [DONE] | [DONE] | |
SoundCloud | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Tumblr | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
[DONE] | [DONE] | [DONE] | [DONE] | [DONE] | |
Yahoo | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Youtube | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
BT24 | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Google Mail | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
DuckDuckGo | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
MSN | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
PayPal | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Reuters | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
BBC News | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Moztrap | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Marketplace | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Mozilla Wiki | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Booking.com | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
London Stock Exchange | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Moneybookers | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Oracle Academy | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Sports Direct | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Emag | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Imdb | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Aol | [DONE] | [DONE] | [DONE] | [DONE] | [DONE] |
Issues Found:
- No issues found
Fix Verifications
- [DONE] bug 929068 Tumblr logo does not show correctly at first load. (style sheet would not load at first load)
- [DONE] bug 929524 Layout is broken on twitter.com at initial loading
- [DONE] bug 927901 Test has failed: Permission denied to access property 'encrypt'
- [DONE] bug 930827 Hang when visiting icu-project.org
- bug 927975 Enable Firefox /whatsnew for /pl (polish) locale on Firefox 25 -- once updates are released
25.0.1
Build Info
- Builds: build1
- Build ID: 20131112160018
- Change Log: build1
- Mozmill Configs
Check List
- [DONE] Automation (report)
- [DONE] Regression Testing (see below)
- [DONE] Updates on betatest channel (report)
- [DONE] Updates on releasetest channel (report)
- [DONE] Updates on release channel (report)
- This was done via manual spot checks due to problems with the Mozmill infrastructure.
- [DONE] Verify what's new page for 25.0.1-pl via bug 927975
Regression Testing
The following areas have been identified for regression testing and will be covered via Moztrap smoketests.
- Past Regressions
- Focus: Breakpad
- Platforms: Windows XP, Windows 7 64-bit, Windows 8.1 32-bit, Mac OSX 10.9, Ubuntu 13.04 64-bit
- Instructions: Force a crash with the crashme extension and verify that the report is successfully submitted
- SSL Topsite Testing
- Navigate to a website which uses SSL and do some exploratory testing on that site
- Be sure to log in to the website if it has that functionality
- Report any issues with rendering and general usage, in particular if you get any SSL errors
- Suggestions for sites to test include email, banking, social networking, shopping, video streaming, and forum websites
- Try to test at least 25 different websites
- Add PASS if there were no problems or FAIL if you encountered an issue
- Be sure to note any issues in the Issues Found section below
Sites Tested | Windows XP | Windows 7 64-bit | Windows 8.1 32-bit | Mac OSX 10.9 | Ubuntu 13.04 64-bit |
1. https://www.facebook.com/ | PASS | PASS | PASS | PASS | PASS |
2. http://www.youtube.com/ | PASS | PASS | PASS | PASS | PASS |
3. https://login.yahoo.com/ | PASS | PASS | PASS | PASS | PASS |
4. https://www.linkedin.com/ | PASS | PASS | PASS | PASS | PASS |
5. https://login.live.com/ | PASS | PASS | PASS | PASS | PASS |
6. https://twitter.com/ | PASS | PASS | PASS | PASS | PASS |
7. http://www.amazon.com/ | PASS | PASS | PASS | PASS | PASS |
8. https://accounts.google.com/ | PASS | PASS | PASS | PASS | PASS |
9. http://wordpress.com/ | PASS | PASS | PASS | PASS | PASS |
10. http://www.ebay.com/ | PASS | PASS | PASS | PASS | PASS |
11. https://www.tumblr.com/ | PASS | PASS | PASS | PASS | PASS |
12. https://www.pinterest.com/ | PASS | PASS | PASS | PASS | PASS |
13. https://www.paypal.com/ | PASS | PASS | PASS | PASS | PASS |
14. https://instagram.com/accounts/login/ | PASS | PASS | PASS | PASS | PASS |
15. https://login.alibaba.com/ | PASS | PASS | PASS | PASS | PASS |
16. https://www.adcash.com/en/index.php | PASS | PASS | PASS | PASS | PASS |
17. https://vimeo.com/ | PASS | PASS | PASS | PASS | PASS |
18. http://www.aliexpress.com/ | PASS | PASS | PASS | PASS | PASS |
19. https://ib.btrl.ro/BT24/ | PASS | PASS | PASS | PASS | PASS |
20. https://www.pcgarage.ro | PASS | PASS | PASS | PASS | PASS |
21. https://www.emag.ro/ | PASS | PASS | PASS | PASS | PASS |
22. https://en.wikipedia.org/wiki/Main_Page | PASS | PASS | PASS | PASS | PASS |
23. https://soundcloud.com/ | PASS | PASS | PASS | PASS | PASS |
24. http://www.booking.com/ | PASS | PASS | PASS | PASS | PASS |
25. http://www.bbc.com | PASS | PASS | PASS | PASS | PASS |
26. http://eu.battle.net/en/ | PASS | PASS | PASS | PASS | PASS |
27. https://mail.google.com/ | PASS | PASS | PASS | PASS | PASS |
28. https://plus.google.com/ | PASS | PASS | PASS | PASS | PASS |
29. https://quality.mozilla.org | PASS | PASS | PASS | PASS | PASS |
30. http://www.foxnews.com | PASS | PASS | PASS | PASS | PASS |
add more if time permits |
Issues Found:
- No issues found
Fix Verifications
- [DONE] bug 933733 Page doesn't load/render if the mouse is not moving -- verified by reporter
- [DONE] bug 935959 Update Mozilla to NSS 3.15.3 (new alternative NSS branch) to pick up a few fixes
Bugs Reported
- [ON TRACK] bug 938075 Size of some Firefox 25.0.1 win32 l10n candidate builds is 45MB-72MB
Bugs
Potential Blockers
[ON TRACK]
- no tracked bugs
[DONE]
- bug 921457 [WebAudio] No sound is played when connecting a Delay Node
- bug 904620 [WebGL] Hangs in SurfaceStream_TripleBuffer_Async::WaitForCompositor
- bug 929068 Tumblr logo does not show correctly at first load. (style sheet would not load at first load)
- bug 929617 Validity period for stapled OCSP responses is too short
- bug 929524 Layout is broken on twitter.com at initial loading
- bug 927975 Enable Firefox /whatsnew for /pl (polish) locale on #Firefox25
[DROPPED]
- bug 923319 [WebAudio] Performance is extremely poor
- bug 923106 [WebAudio] Sound is not changed in the Oscillator audio source node
- bug 923058 [WebAudio] Jerky sound on Live Input node
Issues Found in Testing
Add any new issues found or old issues reopened to the list below. These issues will be triaged and escalated by the release leads.
33 Total; 5 Open (15.15%); 22 Resolved (66.67%); 6 Verified (18.18%);
Top Crashes
Conduct daily and weekly stability checks and list any tracked bugs below. Any crash in the top-20 should have the topcrash keyword added and be nominated to track. Use bughunter to help with URL correlations.
[ON TRACK]
- no tracked bugs
[DONE]
- bug 929009 crash in `anonymous namespace::downmix_to_stereo<float>(float*, long, float*, int)
- bug 925459 crashes related to bitguard.dll
- bug 921323 crash in OpenGL@0x32af (MakeCurrent from OMTC)
- bug 920725 crash in nsHtml5StreamParser::WriteStreamBytes
- bug 898156 crash in mozilla::net::FTPChannelParent::OnStartRequest
- bug 881636 crash in mozilla::a11y::DocAccessible::UpdateTree
- bug 873733 crash in npjp2 @ mozilla::plugins::parent::_retainobject with Java SE7
- bug 800347 crash in mozilla::ipc::RPCChannel::OnMaybeDequeueOne
[AT RISK]
- no bugs at risk
[DROPPED]
- bug 928020 crash in mozalloc_abort(char const* const) | xul.dll@... | js::Invoke(JSContext*, JS::Value const&, JS::Value const&, unsigned int, JS::Value*, JS::MutableHandle<JS::Value>)
- bug 927081 crash in js::gc::IsAboutToBeFinalized<js::Shape>
- bug 925407 crash in npswf32_11_9_900_117.dll@0x345b8d
- bug 812683 [Win7] startup crash in _d2d_clear_surface @ CContext::RestorePipelineStateImpl (d3d11.dll, KB2670838, Asus Gamer OSD)
- bug 767343 crash in nsSupportsStringImpl::SetData with abort
QA Wanted
[ON TRACK]
- no bugs (list)
[DONE]
- bug 929063 Twitter layout broken/missing items
- bug 918612 [Mac] Firefox 24 doesn't start when home directory is on AFP network share
- bug 913407 Rendering issue in Addon manager
- bug 913043 [adbe 3627468] Flash 11.9 crash in F1987069282
- bug 922875 Stencil test fails even if there is no stencil buffer in the current FBO
[DROPPED]
- bug 919076 Saved files permissions set incorrectly on Linux and Mac
- bug 917011 Video on second screen gets displayed on first screen in fullscreen mode
- bug 886670 Crash in mozilla::gl::SharedSurface_GLTexture::Fence @ libGPUSupportMercury on citadel demo guided tour
- bug 698296 crash in js::SweepBackgroundThings @ js::gc::Arena::finalize
Verifications
- How to Verify a Fix
- Pick a bug from this list
- Download a build that is known to reproduce the bug and test to make sure you can see the bug
- Download a build that is said to be fixed and test to make sure you cannot see the bug
- If the bug reproduces in the fixed build:
- set the STATUS field to REOPENED and provide a comment detailing your testing
- If the bug does not reproduce in the fixed build:
- set the status flag to verified for the version you tested
- set the STATUS field to VERIFIED FIXED if the version tested is the same as the TARGET MILESTONE
- any bugs marked verified will automatically appear in the Verified Fixes section below
- Verified Fixes
272 Total; 272 Open (100%); 0 Resolved (0%); 0 Verified (0%);
Unconfirmed
Pick a bug from this list and try to reproduce the issue using the information provided. If you reproduce the issue, provide a description of the circumstances, set the status to NEW, and move the bug into a component which matches the issue (ie. a bookmarks issue would go in the Bookmarks component). If you can't reproduce the issue ask the reporter for more information by setting the need-info? flag to their email address.
- Confirmed Bugs
187 Total; 187 Open (100%); 0 Resolved (0%); 0 Verified (0%);
Other Bugs
The following other queries should be triaged periodically, mostly by the QA Driver, and nominated for testing.
- regressionwindow-wanted: nominate for qawanted if testing is needed or remove regressionwindow-wanted if the request is no longer valid
- [ fixed]: nominate for verifyme if QA testing is needed, [easy verify] of community testing is needed, or [qa-] if no testing is needed
New Platform Readiness
Windows 8.1
Status | COMPLETE (due 16 October 2013) |
Owner | Anthony Hughes |
Manual Testing | COMPLETE (see known bugs listed below) |
Automated Testing | COMPLETE (see known bugs listed below) |
Bugs |
[ON TRACK]
[DONE]
|
Mac OSX 10.9
Status | COMPLETE (due TBD October 2013) |
Owner | Marcia Knous |
Manual Testing | COMPLETE (see known bugs listed below) |
Automated Testing | COMPLETE (integrated into Mozmill-CI workflow) |
Bugs |
[ON TRACK]
[DONE]
|
Post-mortem
- Agenda & Notes
- Follow-up:
- OCSP - not tested well enough due to lack of live sites to test against
- QA needs to be able to test features even without live use cases
- Should Mozilla try to find a "launch partner" for these types of features in the future?
- WebCrypto - undocumented uplift to Firefox 25
- QA needs to be "involved" in uplift approvals, attending the RelMan meetings is one way we can achieve this
- App Manager - flew under the radar completely
- QA needs to escalate these "failures" as soon as they happen
- Talk to dkl or glob about identifying "features" based on Bugzilla heuristics (patch size, number of files, component, etc)
- l10n (24esr) - RelMan's mistake
- QA is incapable of identifying these issues via manual testing and automation
- RelEng should harden their system to mitigate exposure to human error
- QA should try to involve l10n more in community events, reach out to localizers more
- Mozmill-CI Instability
- Cannot continue to force European staff to troubleshoot when they are on PTO, sick, or should be sleeping
- A-Team needs someone in a North American timezone to maintain and troubleshoot issues during Beta/Release pushes
- OCSP - not tested well enough due to lack of live sites to test against