MozillaWiki

Releases/Firefox 10 ESR/Test Plan

< Releases
Revision as of 13:38, 23 April 2012 by Teodosia.pop (talk | contribs) (→‎Manual Tests)

Firefox 10 ESR

The following is a test plan to sign-off Firefox 10 ESR for Release.

Firefox 10.0

Builds: FTP, HTTP

Checklist
  • [DONE] Functional automated tests, synced to Litmus
  • [DONE] Manual smoketests
  • [DONE] Bug fix verifications (N/A for first ESR)
  • [DONE] Update tests (N/A for first ESR)

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Assignee Result: report
Windows 2000 ashughes PASS - 2012-01-30
Windows XP ashughes PASS - 2012-01-30
Windows Vista ashughes PASS - 2012-01-30
Windows 7 32-bit ashughes PASS - 2012-01-30
Windows 7 64-bit ashughes PASS - 2012-01-30
Mac OS X 10.6 ashughes PASS - 2012-01-30
Mac OS X 10.7 ashughes PASS - 2012-01-30
Ubuntu 11.10 32-bit ashughes PASS - 2012-01-30
Ubuntu 11.10 64-bit ashughes PASS - 2012-01-30
Test Failures

Manual Tests

The following are the detailed results of the manual tests.

Platform Assignee Smoketests
Windows 2000 Simona PASS - 2012-01-31
Windows XP Ioana PASS - 2012-01-31
Windows Vista Simona PASS - 2012-01-31
Windows 7 32-bit Mihaela PASS - 2012-01-31
Windows 7 64-bit Mihaela PASS - 2012-01-31
Mac OS X 10.6 VladG PASS - 2012-01-31
Mac OS X 10.7 VladG PASS - 2012-01-31
Ubuntu 11.10 32-bit Ioana PASS - 2012-01-31
Ubuntu 11.10 64-bit Virgil PASS - 2012-01-31

Litmus: testrun, results

Exploratory Testing
  • Check top-sites like Facebook, Youtube, Gmail, Amazon, banking websites, audio/video streaming, etc
  • Check a random assortment of top add-ons to ensure compatibility
    • The tests and results are listed in the Etherpad

Known Issues

Bug ID Summary Reporter Status
bug 719380 Html5 video flickers in full screen mode VladG NEW
bug 520600 Switching into full screen video playback mode while video is paused switches into play mode automatically Henrik Skupin NEW
bug 702894 Using esc to exit full screen video from built-in player breaks player in pre-DOM API full-screen implementation Christopher Blizzard ASSIGNED
bug 585816 no thumbnail when dragging tabs Juan Becerra RESOLVED WONTFIX
bug 671535 Videos fail to play on Windows 2000 Simona B RESOLVED WONTFIX
bug 382124 Crash Reporter (= Breakpad) doesn't write minidumps on Windows 2000 (old DbgHelp.dll / MiniDumpWriteDump()) Steve England RESOLVED WONTFIX

Firefox 10.0.1

Builds: FTP, HTTP
Litmus: testrun, report

Checklist
  • [DONE] Functional automated tests, synced to Litmus
  • [DONE] Manual tests
  • [DONE] Bug fix verifications
  • [ON TRACK] Update tests on releasetest
  • [ON TRACK] Update tests on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Assignee Result: report
Windows 2000 ashughes PASS - 2012-02-08
Windows XP ashughes PASS - 2012-02-08
Windows Vista ashughes PASS - 2012-02-08
Windows 7 32-bit ashughes PASS - 2012-02-08
Windows 7 64-bit ashughes PASS - 2012-02-08
Mac OS X 10.6 ashughes PASS - 2012-02-08
Mac OS X 10.7 ashughes PASS - 2012-02-08
Ubuntu 11.10 32-bt ashughes PASS - 2012-02-08
Ubuntu 11.10 64-bit ashughes PASS - 2012-02-08
Test Failures
  • testSecurity/testSafeBrowsingWarningPages (known, bug 705182)
  • testSecurity/testSubmitUnencryptedInfoWarning (bug 725486)

Manual Tests

The following are the detailed results of the manual testing.

Smoketests

Run the manual smoketests and report results below.

Platform Assignee Result
Windows 2000 Simona PASS with known issues - 2012-02-09
Windows XP Ioana PASS - 2012-02-09
Windows Vista Simona PASS with known issues - 2012-02-09
Windows 7 32-bit Mihaela PASS with known issues - 2012-02-09
Windows 7 64-bit Paul PASS with known issues - 2012-02-09
Mac OS X 10.6.8 Mihaela PASS - 2012-02-09
Mac OS X 10.7.2 Virgil PASS - 2012-02-09
Ubuntu 11.10 32-bit Ioana PASS - 2012-02-09
Ubuntu 11.10 64-bit Virgil PASS - 2012-02-09

Results in Litmus

Update Tests

The following are details of update verifications

Platform Releasetest: report Release: report
Windows 2000 PASS - 2012-02-09 PASS - 2012-02-10
Windows XP PASS - 2012-02-09 PASS - 2012-02-10
Windows Vista PASS - 2012-02-09 PASS - 2012-02-10
Windows 7 32-bit PASS - 2012-02-09 PASS - 2012-02-10
Windos 7 64-bit PASS - 2012-02-09 PASS - 2012-02-10
Mac OS X 10.6 PASS - 2012-02-09 PASS - 2012-02-10
Mac OS 10.7 PASS - 2012-02-09 PASS - 2012-02-10
Ubuntu 11.10 32-bit PASS - 2012-02-09 PASS - 2012-02-10
Ubuntu 11.10 64-bit PASS - 2012-02-09 PASS - 2012-02-10
Issues

Bug Fix Verifications

Bug Status
bug 718939
Java applet causes text entry fields to become semi-unresponsive
TIP: See comment 23 and 25 for steps to reproduce 		VERIFIED FIXED (juanb)
bug 724284
Use after free in nsXBLDocumentInfo::ReadPrototypeBindings 		VERIFIED FIXED (marcia)

Issues Found

Bug Notes
bug 725631
Forward button disabled by visiting Youtube before other websites 		DUPE of bug 682115?
not widely reproducible
bug 725614
Keyboard shortcuts fail on pages containing Java/Shockwave/PDF 		Not a regression, reproducible back to 3.6.26
bug 725306
Vimeo doesn't enter full screen correctly via context menu 		Regressed in Firefox 7.0a1 2011-07-28
bug 725296
Browser is unresponsive while Silverlight content is loading 		Not a regression, affects Chrome
bug 709029
RealPlayer plugin crashes when trying to play a video 		Not a regression, Firefox 4.0 (OOPP?)
bug 638770
Clicking on a quicktime video pauses it 		Trivial UX issue, not worth tracking
bug 78414
Keyboard shortcuts fail when plug-in has focus 		Very old bug, no reason to start blocking
bug 692446
Firefox prompts to install windows media player plugin even though it is already installed 		MSI bug

Firefox 10.0.2 ESR

Builds: FTP, HTTP
Litmus: testrun, report
Mozmill: [ report]

Checklist
  • [DONE] Functional automated tests, synced to Litmus
  • [DONE] Manual smoketests
  • [DONE] Bug fix verifications
  • [DONE] Focused tests
  • [DONE] Update tests on releasetest
  • {{ |}} Get final sign-off from Bob Moss
  • {{ |}} Update tests on release

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Result: report
Windows 2000 PASS - 2012-02-16
Windows XP PASS - 2012-02-16
Windows Vista PASS - 2012-02-16
Windows 7 32-bit PASS - 2012-02-16
Windows 7 64-bit PASS - 2012-02-16
Mac OS X 10.6 PASS - 2012-02-16
Mac OS X 10.7 PASS - 2012-02-16
Ubuntu 11.10 32-bit PASS - 2012-02-16
Ubuntu 11.10 64-bit PASS - 2012-02-16
Test Failures

Manual Tests

The following are the detailed results of the manual Smoke and ESR tests.

Platform Assignee Result: report
Windows 2000 Simona PASS - 2012-02-17
Windows XP RemusPop PASS - 2012-02-17
Windows Vista Simona PASS - 2012-02-17
Windows 7 32-bit AlinT PASS - 2012-02-17
Windows 7 64-bit AlinT PASS - 2012-02-17
Mac OS X 10.6.8 AlexLakatos PASS - 2012-02-17
Mac OS X 10.7.2 AlexLakatos PASS - 2012-02-17
Ubuntu 11.10 32-bit RemusPop PASS - 2012-02-17
Ubuntu 11.10 64-bit Vlad Maniac PASS - 2012-02-17

Focus Tests

  1. [DONE] PNG handling exploratory testing Etherpad

Update Tests

The following are details of update verifications

Platform Releasetest: report Esr: report
Windows 2000 PASS - 2012-02-17 PASS - 2012-02-17
Windows XP PASS - 2012-02-17 PASS - 2012-02-17
Windows Vista PASS - 2012-02-17 PASS - 2012-02-17
Windows 7 32-bit PASS - 2012-02-17 PASS - 2012-02-17
Windows 7 64-bit PASS - 2012-02-17 PASS - 2012-02-17
Mac OS X 10.6 PASS - 2012-02-17 PASS - 2012-02-17
Mac OS 10.7 PASS - 2012-02-17 PASS - 2012-02-17
Ubuntu 11.10 32-bit PASS - 2012-02-17 PASS - 2012-02-17
Ubuntu 11.10 64-bit PASS - 2012-02-17 PASS - 2012-02-17
Issues

Bug Fix Verifications

[DONE] bug 727401 libpng: integer overflow leading to heap-buffer overflow

Issues Found

Bug Status
bug 728189
Drag a large png causes firefox to hang 		NEW
bug 728200
Dragging a large png image from browser to desktop temporary hangs 		NEW
bug 728202
Inconsistent behavior when dragging an image to an application 		NEW

Firefox 10.0.3 ESR

Respinning due to bug 733012

Builds: FTP, HTTP
Litmus: testrun, report
Mozmill: [ report]

Checklist
  • [DONE] Functional automated tests, synced to Litmus
  • [DONE] Manual smoketests
  • [DONE] Focus Tests
  • [ON TRACK] Bug fix verifications
  • {{ |}} Update tests on releasetest
  • {{ |}} Get final sign-off from Bob Moss
  • {{ |}} Update tests on esr

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Result: report
Windows 2000 PASS - 2012-03-03
Windows XP PASS - 2012-03-03
Windows Vista PASS - 2012-03-03
Windows 7 32-bit PASS - 2012-03-03
Windows 7 64-bit PASS - 2012-03-03
Mac OS X 10.6 PASS - 2012-03-03
Mac OS X 10.7 PASS - 2012-03-03
Ubuntu 11.10 32-bit PASS - 2012-03-03
Ubuntu 11.10 64-bit PASS - 2012-03-03
Test Failures
  • testPreferredLanguage.js::testSetLanguages (bug 731158)
  • testSafeBrowsingWarningPages.js::testWarningPages (bug 705182)

Manual Tests

The following are the detailed results of the manual Smoke and ESR tests.

Platform Assignee Result: report
Windows 2000 Simona PASS - 2012-03-05
Windows XP Paul PASS - 2012-03-05
Windows Vista Simona PASS - 2012-03-05
Windows 7 32-bit Mihaela PASS - 2012-03-05
Windows 7 64-bit Paul PASS - 2012-03-05
Mac OS X 10.6.8 Vlad PASS - 2012-03-05
Mac OS X 10.7.3 Vlad PASS - 2012-03-05
Ubuntu 11.10 32-bit Virgil PASS - 2012-03-05
Ubuntu 11.10 64-bit Virgil PASS - 2012-03-05

Focus Tests

The following are the detailed results of the focused testing and running the regression tests

Platform Assignee Result: report
Windows 7 32-bit Mihaela PASS - 2012-03-05
Windows XP Paul PASS - 2012-03-05
Mac OS X 10.6.8 VladG PASS - 2012-03-05
Ubuntu 11.10 64-bit Virgil PASS - 2012-03-05

Update Tests

The following are details of update verifications

Platform Releasetest: [ report] Esr: [ report]
Windows 2000 result - YYYY-MM-DD result - YYYY-MM-DD
Windows XP result - YYYY-MM-DD result - YYYY-MM-DD
Windows Vista result - YYYY-MM-DD result - YYYY-MM-DD
Windows 7 32-bit result - YYYY-MM-DD result - YYYY-MM-DD
Windows 7 64-bit result - YYYY-MM-DD result - YYYY-MM-DD
Mac OS X 10.6 result - YYYY-MM-DD result - YYYY-MM-DD
Mac OS 10.7 result - YYYY-MM-DD result - YYYY-MM-DD
Ubuntu 11.10 32-bit result - YYYY-MM-DD result - YYYY-MM-DD
Ubuntu 11.10 64-bit result - YYYY-MM-DD result - YYYY-MM-DD
Issues
  • bug 733012 - Firefox 10.0.3esr#1 update channel defaults to nightly-esr10 instead of esr

Bug Fix Verifications

Check that the fixes ported to Firefox 10.0.3 ESR have indeed been fixed. A bug which is reproducible in Firefox 10.0.2 ESR but not in Firefox 10.0.3 ESR is fixed. When verifying a fix, set the status-firefox-esr10 flag to verified.

Fixes landed for Firefox 10.0.3 ESR: bugs (34 / 34 TRIAGED)

Softvision verifications etherpad

Issues Found

Bug Status
bug 725614 - Shortcut keys fail on pages containig Java/Shockwave/PDF after unfocus Previously known, not tracking for 10.0.3ESR
bug 725296 - Browser is unresponsive while Silverlight content is loading Previously known, not tracking for 10.0.3ESR
bug 729872 - No text cursor after HTML5 fullscreen window deactivation Previously known, tracking to fix in Firefox 13
bug 78414 - Shortcut keys fail when plug-in (flash, acrobat, quicktime) has focus Previously known, not tracking for 10.0.3ESR
bug 731981 - Missing steps for missing Shockwave plugin notification Tech Evangelism, not a Firefox bug
bug 709029 - The RealPlayer plugin crashes when trying to play a RealPlayer video Previously known, DUPE of bug 667399
bug 590963 - Google Maps - Earth View Causes Freeze Previously know, not tracking for 10.0.3ESR

Firefox 10.0.3 ESR build 2

Respin due to bug 733012

Builds: FTP, HTTP
Litmus: testrun, report
Mozmill: [ report]

Checklist
  • [DONE] Functional automated tests, synced to Litmus
  • [DONE] Manual smokestests and ESR tests
  • [DONE] Update tests on releasetest
  • [DONE] Get final sign-off from Bob Moss
  • {{ |}} Update tests on esr

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Result: report
Windows 2000 PASS - 2012-03-09
Windows XP PASS - 2012-03-09
Windows Vista PASS - 2012-03-09
Windows 7 32-bit PASS - 2012-03-09
Windows 7 64-bit PASS - 2012-03-09
Mac OS X 10.6 PASS - 2012-03-09
Mac OS X 10.7 PASS - 2012-03-09
Ubuntu 11.10 32-bit PASS - 2012-03-09
Ubuntu 11.10 64-bit PASS - 2012-03-09
Test Failures

Manual Tests

The following are the detailed results of the manual Smoke and ESR tests.

Platform Assignee Result: report
Windows 2000 Simona PASS - 2012-03-12
Windows XP Ioana PASS - 2012-03-12
Windows Vista Simona PASS - 2012-03-12
Windows 7 32-bit Mihaela PASS - 2012-03-12
Windows 7 64-bit Paul PASS - 2012-03-12
Mac OS X 10.6.8 VladG PASS - 2012-03-12
Mac OS X 10.7.3 VladG PASS - 2012-03-12
Ubuntu 11.10 32-bit Ioana PASS - 2012-03-12
Ubuntu 11.10 64-bit Virgil PASS - 2012-03-12

Update Tests

The following are details of update verifications

Platform Releasetest: report Esr: report
Windows 2000 PASS - 2012-03-12 PASS - 2012-03-13
Windows XP PASS - 2012-03-12 PASS - 2012-03-13
Windows Vista PASS - 2012-03-12 PASS - 2012-03-13
Windows 7 32-bit PASS - 2012-03-12 PASS - 2012-03-13
Windows 7 64-bit PASS - 2012-03-12 PASS - 2012-03-13
Mac OS X 10.6 PASS - 2012-03-12 PASS - 2012-03-13
Mac OS 10.7 PASS - 2012-03-12 PASS - 2012-03-13
Ubuntu 11.10 32-bit PASS - 2012-03-12 PASS - 2012-03-13
Ubuntu 11.10 64-bit PASS - 2012-03-12 PASS - 2012-03-13
Issues

Issues Found

Bug Status
bug 711742 - HTML5 "Save Video As" saves empty .webm if not played first QA not tracking -- devs investigating
bug 734837 - Broken history when navigating to and from cnn.com QA investigating -- might be server-side issue
bug 734866 - Timeline is still seen in HTML5 full screen mode INVALID - not a Firefox bug

Firefox 10.0.4 ESR

Builds: FTP, HTTP
Litmus: testrun, report
Mozmill: functional, updates
Build ID: 20120420145309
Changeset: 64dad3d2e86e

Checklist
  • [DONE] Functional automation
  • {{ |}} Sync automation to Litmus
  • {{ |}} Manual smoketests
  • {{ |}} Focus Tests
  • {{ |}} Bug fix verifications
  • [DONE] Update tests on releasetest
  • {{ |}} Get final sign-off from Bob Moss
  • {{ |}} Update tests on esr

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Result: report
Windows 2000 PASS - 2012-04-22
Windows XP PASS - 2012-04-22
Windows Vista PASS - 2012-04-22
Windows 7 32-bit PASS - 2012-04-22
Windows 7 64-bit PASS - 2012-04-22
Mac OS X 10.6 PASS - 2012-04-22
Mac OS X 10.7 PASS - 2012-04-22
Ubuntu 11.10 32-bit PASS - 2012-04-22
Ubuntu 11.10 64-bit PASS - 2012-04-22
Test Failures

Manual Tests

The following are the detailed results of the manual Smoke and ESR tests.

Platform Assignee Result: report
Windows 2000 Mihaela In Progress - 2012-04-23
Windows XP Ioana PASS - 2012-04-23
Windows Vista Simona In Progress - 2012-04-23
Windows 7 32-bit Simona PASS - 2012-04-23
Windows 7 64-bit Teodosia PASS - 2012-04-23
Mac OS X 10.6.8 VladG result - YYYY-MM-DD
Mac OS X 10.7.3 Alex result - YYYY-MM-DD
Ubuntu 11.10 32-bit Simona PASS - 2012-04-23
Ubuntu 11.10 64-bit Virgil result - YYYY-MM-DD

Focus Tests

The following are the detailed results of the focused testing and running the regression tests

Platform Assignee Result: report
Windows 7 32-bit Alin result - YYYY-MM-DD
Windows XP Simona result - YYYY-MM-DD
Mac OS X 10.6.8 Florin result - YYYY-MM-DD
Ubuntu 11.10 32-bit Remus result - YYYY-MM-DD

Update Tests

The following are details of update verifications

Platform Releasetest: report Esr: [ report]
Windows 2000 PASS - 2012-04-22 result - YYYY-MM-DD
Windows XP PASS - 2012-04-22 result - YYYY-MM-DD
Windows Vista PASS - 2012-04-22 result - YYYY-MM-DD
Windows 7 32-bit PASS - 2012-04-22 result - YYYY-MM-DD
Windows 7 64-bit PASS - 2012-04-22 result - YYYY-MM-DD
Mac OS X 10.6 PASS - 2012-04-22 result - YYYY-MM-DD
Mac OS 10.7 PASS - 2012-04-22 result - YYYY-MM-DD
Ubuntu 11.10 32-bit PASS - 2012-04-22 result - YYYY-MM-DD
Ubuntu 11.10 64-bit PASS - 2012-04-22 result - YYYY-MM-DD
Manual Spotchecks
  • What's New Page
  • First Run Page
Issues

Bug Fix Verifications

Check that the fixes ported to Firefox 10.0.4 ESR have indeed been fixed. A bug which is reproducible in Firefox 10.0.3 ESR but not in Firefox 10.0.4 ESR is fixed. When verifying a fix, set the status-firefox-esr10 flag to verified.

Fixes landed for Firefox 10.0.4 ESR: bugs (6 / 24 triaged)

Softvision verifications etherpad

Issues Found

Full Query
ID Summary Priority Status
678482 Scrolling in web pages with java and shockwave scrambles the content -- RESOLVED

1 Total; 0 Open (0%); 1 Resolved (100%); 0 Verified (0%);

Retrieved from "https://wiki.allizom.org/index.php?title=Releases/Firefox_10_ESR/Test_Plan&oldid=423034"