Revision as of 22:49, 31 October 2012

« Firefox 16 | Firefox 17 | Firefox 18 »

Firefox 17 Test Plan

Summary

Milestone	Firefox 17
Status	Currently in Beta -- merging to Released on November 20, 2012
Lead	Anthony Hughes (irc: ashughes)
2nd	Juan Becerra (irc: juanb)
Builds	[ftp] [http] [web]

The following is the test plan for Firefox 17 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 17. After the release this document will be used as an archive of what was done to validate this release.

If you have some free time, please pick a task below and get in touch with one of the leads.

Details

The following is a description of the tasks which need to be completed throughout this release.

2012-07-16: [DONE] Nightly Merge
2012-08-27: [DONE] Aurora Merge
2012-10-08: [DONE] Pre-Beta feature sign-offs
2012-10-11: [DONE] Beta 1
2012-10-18: {{|}} Beta 2
2012-10-25: {{|}} Beta 3
2012-11-01: {{|}} Beta 4
2012-11-08: {{|}} Beta 5
2012-11-15: {{|}} Beta 6
2012-11-19: {{|}} Release Candidate
2012-11-20: {{|}} Final

Nightly

Merge

Functional Automation - result

Update Spotchecks - result

Features

See the Feature Release Tracking wiki for a general overview

Features

See the Feature Release Tracking wiki for a general overview

Feature	Owner	Status	Tests
Improve display of location bar results	Virgil Dicu	On track
Firefox Social Integration	Anthony Hughes	On track
HTML Tree Editor	Mihaela Velimiroviciu
Silent Update not now prompt	Ioana Budnar	Release target removed.

Endurance Automation

The purpose of this section is to track any potential memory regressions via spikes reported in the dashboard.

What to Do

Check the Charts
If you see an anomalous event (ie. spike or dip), get the report for the specific event
If the event happens twice in a row, file a bug under Mozmill Tests
Notify Anthony Hughes of the issue
Record the event in this wiki below

Current Status (Mihaela Velimiroviciu) - Last Updated

Bugzilla query error

Query options must be valid JSON.1

Functional Automation

The purpose of this section is to track any potential functional regressions via results reported in the dashboard.

What to Do

Check the testrun reports
If you see any failing tests, check that there are not any bugs on file for it already
If not, file a bug under Mozmill Tests
Notify Anthony Hughes of the issue
Record the event in this wiki below

Current Status (Vlad Maniac / Alex Lakatos) - Last Updated

Bug Triage

The purpose of this section is to triage UNCONFIRMED incoming bugs.

What to Do

Check the UNCONFIRMED bugs
Test the bugs in the latest Firefox Nightly
If you can't reproduce the bug, ask for the reporter to provide more information
If you reproduce the bug, add your testing information and mark the bug NEW
Notify Anthony Hughes of the NEW issue

Current Status (SoftVision) - Last Updated

Result Spreadsheet

Aurora Merge

NOTE - OSX 10.5 should not be able to install nor update to Firefox 17


Builds	nightly
Changeset	tip (pushlog)

Checklist

[DONE] Automated Testing
[DONE] Manual Testing
[DONE] Update Testing
[DONE] Bug Triage
[DONE] Peer Sign-off
[ON TRACK] Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	PASS: report
Windows XP	PASS - 2012-08-28
Windows Vista	PASS - 2012-08-28
Windows 7 32-bit	PASS - 2012-08-28
Windows 7 64-bit	PASS - 2012-08-28
Windows 8 32-bit	N/A (pending IT)
Windows 8 64-bit	N/A (pending IT)
Mac OS X 10.6	PASS - 2012-08-28
Mac OS X 10.7	PASS - 2012-08-28
Mac OS X 10.8	PASS - 2012-08-28
Ubuntu 12.04 32-bit	PASS - 2012-08-28
Ubuntu 12.04 64-bit	PASS - 2012-08-28

Test Failures (known bugs)

Manual Testing

Operating System	Tester	Plug-in Compatibility	Web Compatibility	OSX 10.5 EOL
Windows XP	Simona	PASS - 2012-08-29	PASS - 2012-08-29	N/A
Windows 7	Ioana	PASS - 2012-08-29	PASS - 2012-08-29	N/A
Windows 8	Paul	PASS - 2012-08-30	PASS - 2012-08-30	N/A
Mac OS X	VladG	PASS - 2012-08-30	PASS - 2012-08-30	PASS - 2012-08-30
Ubuntu 12.04	Ioana	PASS - 2012-08-30	PASS - 2012-08-30	N/A

Plug-in Compatibility

Install the following plug-ins and go to websites which use them, looking for rendering issues, functional issues, performance issues, and crashes.

Adobe Flash Player
- 11.4 Release (make sure you can autoupdate)
Microsoft Silverlight
RealPlayer
QuickTime

Web Compatibility

Test the following websites for rendering issues, functional issues, performance issues, and crashes

Overall Top 10
Top by Country (pick a random site from three random countries)
Top by Category (pick a random site from three random categories)
Webmail clients like gmail, hotmail, and yahoo mail (pay particular attention to any built in web messenger software)
Try Google's MapsGL demo
Try YouTube's HTML5 demo

OSX 10.5 EOL

Verify the following:

Mac OSX 10.5 cannot download Firefox 17 (I'm not sure if this is scoped but try it anyway)
Mac OSX 10.5 cannot start Firefox 17
Mac OSX 10.5 cannot update to Firefox 17

The build can be downloaded and installed
Aurora didn't start, showing a incompatibility message
The update channel seems to be close so I didn't manage to perform an update

Feature Sign-offs

Log in to moztrap.mozilla.org
Select Run Tests
Select Product: Desktop Firefox
Select Versions: 17
Select Runs: Firefox 17 Aurora Merge Feature sign off

Feature	Owner	Win32	Mac	Linux
Improve display of location bar results	Virgil Dicu	PASS	PASS	PASS
Firefox Social Integration	Anthony Hughes	PASS	PASS	PASS

Details

Report any detailed results or bugs here...

Update Testing

Spotcheck updates from a previous Aurora to Aurora 17.

Platform	Auroratest	Aurora
Windows 7 64-bit	PASS - 2012-08-30	PASS - 2012-08-30
Windows 8 64-bit	PASS - 2012-08-30	PASS - 2012-08-30
Mac OS X 10.7	PASS - 2012-08-30	PASS - 2012-08-30
Ubuntu 12.04 64-bit	PASS - 2012-08-30	PASS - 2012-08-30

Issues

Bug Triage

VERIFYME (5): verify the fix and do some exploratory testing around the issue
FIXED (139): triage these bugs and test to ensure the fix is effective
QAWANTED (6): triage these bugs and test to fulfill the QA request
UNCONFIRMED (8): triage these bugs for potential new regressions

Results

VERIFIED: 24/139
WONTVERIFY: 3/139
HELPWANTED: 9/139

Beta 1

Builds	~~#1: FTP, HTTP~~ #2: FTP, HTTP
ID	~~#1: 20121009144933~~ #2: 20121010150351
Changeset	6b83222781e3 (pushlog)

Advisory

RelEng issues in generating the builds (see bug 796959 comment 5)

Checklist

[DONE] Automated Testing
[DONE] Manual Testing
[DONE] Update Testing
[DONE] Bug Triage
[DONE] Peer Sign-off
[DONE] Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	Build #1 Results (report)	Build #2 Results (report)
Windows XP	PASS - 2012-10-09	PASS - 2012-10-10
Windows Vista	PASS - 2012-10-09	PASS - 2012-10-10
Windows 7 32-bit	PASS - 2012-10-09	PASS - 2012-10-10
Windows 7 64-bit	PASS - 2012-10-09	PASS - 2012-10-10
Windows 8 32-bit	PASS - 2012-10-09	PASS - 2012-10-10
Windows 8 64-bit	PASS - 2012-10-09	PASS - 2012-10-10
Mac OS X 10.6	PASS - 2012-10-09	PASS - 2012-10-10
Mac OS X 10.7	PASS - 2012-10-09	PASS - 2012-10-10
Mac OS X 10.8	PASS - 2012-10-09	PASS - 2012-10-10
Ubuntu 12.04 32-bit	PASS - 2012-10-09	PASS - 2012-10-10
Ubuntu 12.04 64-bit	PASS - 2012-10-09	PASS - 2012-10-10

Test Failures (known bugs)

Manual Testing

Run the Desktop Firefox Regression Tests in MozTrap and report the results below. Do some exploratory testing around the testcases to try to exercise some edge-cases.

Platform	Tester	Result
Windows 7	Manuela, Ioana	PASS - 2012-10-11
Windows 8	Virgil	PASS - 2012-10-11
Mac OSX 10.7	Ioana, Mihaela	PASS - 2012-10-11
Mac OSX 10.8	Virgil	PASS - 2012-10-11
Ubuntu 12.04	Manuela	PASS - 2012-10-11
Special Cases
Mac OSX 10.5 de-support	Virgil	PASS - 2012-10-11
Update regression tests (requires updates on releasetest)	ashughes	PASS - 2012-10-11

Update Testing

Partial: 16.0b6, 16.0b5
Complete: <= 16.0b4

Platform	Releasetest (report)	Beta (report)
Windows XP	PASS - 2012-10-11	PASS - 2012-10-11
Windows Vista	PASS - 2012-10-11	PASS - 2012-10-11
Windows 7 32-bit	PASS - 2012-10-11	PASS - 2012-10-11
Windows 7 64-bit	PASS - 2012-10-11	PASS - 2012-10-11
Windows 8 32-bit	PASS - 2012-10-11	PASS - 2012-10-11
Windows 8 64-bit	PASS - 2012-10-11	PASS - 2012-10-11
Mac OS X 10.6	PASS - 2012-10-11	PASS - 2012-10-11
Mac OS X 10.7	PASS - 2012-10-11	PASS - 2012-10-11
Mac OS X 10.8	PASS - 2012-10-11	PASS - 2012-10-11
Ubuntu 12.04 32-bit	PASS - 2012-10-11	PASS - 2012-10-11
Ubuntu 12.04 64-bit	PASS - 2012-10-11	PASS - 2012-10-11

Issues

Feature Sign-off

Feature owners to test and sign-off overall quality of their features following uplift to Beta.

Feature	Owner	Win32	Mac	Linux
Improve display of location bar results	Virgil Dicu	n/a release target should be updated	n/a release target should be updated	n/a release target should be updated
HTML tree editor	Mihaela Velimiroviciu	PASS - 2012-10-11	PASS - 2012-10-11	PASS - 2012-10-11
Firefox Social Integration	Anthony Hughes	n/a - delayed to Beta 2	n/a - delayed to Beta 2	n/a - delayed to Beta 2

Build #2 Verifications

VERIFIED FIXED bug 799952
VERIFIED FIXED bug 720619
RESOLVED FIXED bug 798045 (could not verify because could not be reproduced)

Beta 2

Builds	FTP, HTTP
ID	20121017073013
Changeset	2ba4becf6e35 (pushlog)

Checklist

[DONE] Automated Testing
[DONE] Manual Testing
[ON TRACK] Update Testing
[ON TRACK] Bug Triage
[ON TRACK] Peer Sign-off
[ON TRACK] Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	Results (report)
Windows XP	PASS - 2012-10-17
Windows Vista	PASS - 2012-10-17
Windows 7 32-bit	PASS - 2012-10-17
Windows 7 64-bit	PASS - 2012-10-17
Windows 8 32-bit	PASS - 2012-10-17
Windows 8 64-bit	PASS - 2012-10-17
Mac OS X 10.6	PASS - 2012-10-17
Mac OS X 10.7	PASS - 2012-10-17
Mac OS X 10.8	PASS - 2012-10-17
Ubuntu 12.04 32-bit	PASS - 2012-10-17
Ubuntu 12.04 64-bit	PASS - 2012-10-17

Test Failures (known bugs)

testPrivateBrowsing/testDownloadManagerClosed.js (known)

Manual Testing

Run the Firefox 17b2 Regression Tests in MozTrap and report the results below. Do some exploratory testing around the testcases to try to exercise some edge-cases.

Moztrap > Desktop Firefox > 17 > Firefox 17b2 Regression Tests > Operating System > Platform > Run Tests

Platform	Tester	Result (report)
Windows 7	Manuela	PASS - 2012-10-18
Windows 8	Simona	PASS - 2012-10-18
Mac OSX 10.7	Paul	PASS - 2012-10-18
Mac OSX 10.8	Mihaela	PASS - 2012-10-18
Ubuntu 12.04	Virgil	PASS - 2012-10-18
Special Cases
Mac OSX 10.5 de-support	Virgil	PASS - 2012-10-18
Update regression tests (requires updates on releasetest)	ashughes	PASS - 2012-10-18

Update Testing

Partial: 17.0b1, 16.0b6
Complete: <=16.0b5

Platform	Releasetest (report)	Beta ([ report])
Windows XP	PASS - 2012-10-18	result - YYYY-MM-DD
Windows Vista	PASS - 2012-10-18	result - YYYY-MM-DD
Windows 7 32-bit	PASS - 2012-10-18	result - YYYY-MM-DD
Windows 7 64-bit	PASS - 2012-10-18	result - YYYY-MM-DD
Windows 8 32-bit	PASS - 2012-10-18	result - YYYY-MM-DD
Windows 8 64-bit	PASS - 2012-10-18	result - YYYY-MM-DD
Mac OS X 10.6	PASS - 2012-10-18	result - YYYY-MM-DD
Mac OS X 10.7	PASS - 2012-10-18	result - YYYY-MM-DD
Mac OS X 10.8	PASS - 2012-10-18	result - YYYY-MM-DD
Ubuntu 12.04 32-bit	PASS - 2012-10-18	result - YYYY-MM-DD
Ubuntu 12.04 64-bit	PASS - 2012-10-18	result - YYYY-MM-DD

Manual Spotchecks (due to Jenkins outage)

Update	Windows 7	Windows 8	Mac OSX 10.7	Mac OSX 10.8	Ubuntu 12.04
17.0b1 -> 17.0b2 en-US	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22
17.0b1 -> 17.0b2 de	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22
16.0b6 -> 17.0b2 en-US	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22
16.0b6 -> 17.0b2 zh-TW	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22
16.0b1 -> 17.0b2 en-US	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22
16.0b1 -> 17.0b2 es-ES	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22
15.0b6 -> 17.0b2 en-US	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22
15.0b6 -> 17.0b2 pt-BR	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22

Issues

Feature Sign-off

Feature owners to test and sign-off overall quality of their features following uplift to Beta.

Feature	Owner	Win32	Mac	Linux
Firefox Social Integration	Anthony Hughes	PASS - 2012-10-18	PASS - 2012-10-18	PASS - 2012-10-18

Issues

Server-side issues not blocking 17b2 (should be fixed with next server-side push)
- Like button activity not pushing to website
- Full page loading within panel (friends and notifications button)

Beta 3

Builds	FTP, HTTP
ID	20121023124120
Changeset	f8a6138bcfba (pushlog)

Checklist

[DONE] Automated Testing
[DONE] Manual Testing
[DONE] Update Testing
[DONE] Bug Triage
[DONE] Peer Sign-off
[DONE] Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	Results (report)
Windows XP	PASS - 2012-10-24
Windows Vista	PASS - 2012-10-24
Windows 7 32-bit	PASS - 2012-10-24
Windows 7 64-bit	PASS - 2012-10-24
Windows 8 32-bit	PASS - 2012-10-24
Windows 8 64-bit	PASS - 2012-10-24
Mac OS X 10.6	PASS - 2012-10-24
Mac OS X 10.7	PASS - 2012-10-24
Mac OS X 10.8	PASS - 2012-10-24
Ubuntu 12.04 32-bit	PASS - 2012-10-24
Ubuntu 12.04 64-bit	PASS - 2012-10-24

Test Failures (known bugs)

testPrivateBrowsing/testDownloadManagerClosed.js (known)

Manual Testing

Run the Firefox 17b3 Regression Tests in MozTrap and report the results below. Do some exploratory testing around the testcases to try to exercise some edge-cases.

Moztrap > Desktop Firefox > 17 > Firefox 17b3 Regression Tests > Operating System > Platform > Run Tests

Platform	Tester	Result (report)
Windows 7	Simona	PASS - 2012-10-24
Windows 8	Manuela	PASS - 2012-10-24
Mac OSX 10.7	Virgil	PASS - 2012-10-24
Mac OSX 10.8	Mihaela	PASS - 2012-10-24
Ubuntu 12.04	Ioana	PASS - 2012-10-24
Special Cases
Mac OSX 10.5 de-support	Virgil	PASS - 2012-10-24
Update regression tests (requires updates on releasetest)	ashughes	result - YYYY-MM-DD

Bug Fix Verifications

bug 793969 - verified (mwobensmith)
bug 801305 - verified (mwobensmith)
bug 802557 - verified (mwobensmith)
bug 803583 - delayed to 17b4 (juanb)
bug 803181 - verified (virgil)

Update Testing

Partial: 17.0b2, 17.0b1
Complete: <=16.0b6

Platform	Releasetest (report)	Beta ([ report])
Windows XP	PASS - 2012-10-24	result - YYYY-MM-DD
Windows Vista	PASS - 2012-10-24	result - YYYY-MM-DD
Windows 7 32-bit	PASS - 2012-10-24	result - YYYY-MM-DD
Windows 7 64-bit	PASS - 2012-10-24	result - YYYY-MM-DD
Windows 8 32-bit	PASS - 2012-10-24	result - YYYY-MM-DD
Windows 8 64-bit	PASS - 2012-10-24	result - YYYY-MM-DD
Mac OS X 10.6	PASS - 2012-10-24	result - YYYY-MM-DD
Mac OS X 10.7	PASS - 2012-10-24	result - YYYY-MM-DD
Mac OS X 10.8	PASS - 2012-10-24	result - YYYY-MM-DD
Ubuntu 12.04 32-bit	PASS - 2012-10-24	result - YYYY-MM-DD
Ubuntu 12.04 64-bit	PASS - 2012-10-24	result - YYYY-MM-DD

Issues

Beta 4

Builds	FTP, HTTP
ID	20121031065642
Changeset	486335dcb4a2 (pushlog)

Checklist

{{ |}} Automated Testing
{{ |}} Manual Testing
{{ |}} Update Testing
{{ |}} Bug Triage
{{ |}} Peer Sign-off
{{ |}} Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	Results ([ report])
Windows XP	result - YYYY-MM-DD
Windows Vista	result - YYYY-MM-DD
Windows 7 32-bit	result - YYYY-MM-DD
Windows 7 64-bit	result - YYYY-MM-DD
Windows 8 32-bit	result - YYYY-MM-DD
Windows 8 64-bit	result - YYYY-MM-DD
Mac OS X 10.6	result - YYYY-MM-DD
Mac OS X 10.7	result - YYYY-MM-DD
Mac OS X 10.8	result - YYYY-MM-DD
Ubuntu 12.04 32-bit	result - YYYY-MM-DD
Ubuntu 12.04 64-bit	result - YYYY-MM-DD

Test Failures (known bugs)

testPrivateBrowsing/testDownloadManagerClosed.js (known)

Manual Testing

Run the Firefox 17b4 Regression Tests in MozTrap and report the results below. Do some exploratory testing around the testcases to try to exercise some edge-cases.

Moztrap > Desktop Firefox > 17 > Firefox 17b4 Regression Tests > Operating System > Platform > Run Tests

If you have an AMD or NVIDIA graphics card, please install the Beta drivers during your testing

AMD Catalyst 12.11 Beta (download)
NVidia GeForce 310.33 Beta (download)
keep your eye out for rendering issues and crashes

Platform	Tester	Result (report)
Windows 7	tester	result - YYYY-MM-DD
Windows 8	tester	result - YYYY-MM-DD
Mac OSX 10.7	tester	result - YYYY-MM-DD
Mac OSX 10.8	tester	result - YYYY-MM-DD
Ubuntu 12.04	tester	result - YYYY-MM-DD
Special Cases
Mac OSX 10.5 de-support	tester	result - YYYY-MM-DD
Update regression tests (requires updates on releasetest)	ashughes	result - YYYY-MM-DD

Additional Manual Spotchecks

Due to automation infrastructure issues we will conduct the following manual spotchecks before signing off the Beta.

Test Area	Windows 7 64-bit	Windows 8 32-bit	Windows 8 64-bit
Bookmarks	result (tester)	result (tester)	result (tester)
Private Browsing	result (tester)	result (tester)	result (tester)
Session Restore	result (tester)	result (tester)	result (tester)
Tabbed Browsing	result (tester)	result (tester)	result (tester)
Add-ons & Themes	result (tester)	result (tester)	result (tester)

Test Area Details

Bookmarks
- Add, Remove, Reorder, Tag, Keyword, Live (RSS)
Private Browsing
- Authenticated session, cookies, cache, history
Session Restore
- Normal restart, Crash, Multiple crash
Tabbed Browsing
- Pinned tabs, Panorama, Tab scrolling, Tab reordering, Multi-window, Multi-monitor
Add-ons & Themes
- Install several add-ons and themes from AMO and third-party, verify their functionality

Bug Fix Verifications

bug 803583 - status (juanb)
bug 776247 - status (jsmith)
bug 806991 - status (ashughes)

Update Testing

Partial: 17.0b3, 17.0b2
Complete: <=17.0b1

Platform	Releasetest ([ report])	Beta ([ report])
Windows XP	result - YYYY-MM-DD	result - YYYY-MM-DD
Windows Vista	result - YYYY-MM-DD	result - YYYY-MM-DD
Windows 7 32-bit	result - YYYY-MM-DD	result - YYYY-MM-DD
Windows 7 64-bit	result - YYYY-MM-DD	result - YYYY-MM-DD
Windows 8 32-bit	result - YYYY-MM-DD	result - YYYY-MM-DD
Windows 8 64-bit	result - YYYY-MM-DD	result - YYYY-MM-DD
Mac OS X 10.6	result - YYYY-MM-DD	result - YYYY-MM-DD
Mac OS X 10.7	result - YYYY-MM-DD	result - YYYY-MM-DD
Mac OS X 10.8	result - YYYY-MM-DD	result - YYYY-MM-DD
Ubuntu 12.04 32-bit	result - YYYY-MM-DD	result - YYYY-MM-DD
Ubuntu 12.04 64-bit	result - YYYY-MM-DD	result - YYYY-MM-DD

Issues

Bugs

Triage

VERIFYME: verify the fix and do some exploratory testing around the issue
FIXED: triage these bugs and test to ensure the fix is effective
QAWANTED: triage these bugs and test to fulfill the QA request
UNCONFIRMED: triage these bugs for potential new regressions

New Issues

Full Query

ID	Summary	Priority	Status
772837	Content of drop-down lists like the history in location bar blinks on secondary monitor with hardware acceleration enabled	--	RESOLVED
799856	crash in mozilla::InternalInvalidateThebesLayersInSubtree , Firefox17b1 crashes when I canceled login to Facebook	--	VERIFIED
800389	crash in hang [@ hang \| libpthread-2.15.so@0xd5a1 ]	--	VERIFIED
803057	HTML5 videos continues to play after Pop Out	--	RESOLVED
803074	Rendering is broken after restoring skew() support	--	VERIFIED

5 Total; 0 Open (0%); 2 Resolved (40%); 3 Verified (60%);

Known Issues

Full Query

ID	Summary	Priority	Status
468855	Bringing up a context-menu on a Flash video player pauses the video	--	RESOLVED
638796	The print button/menu item should be disabled for full-page plugins	P4	RESOLVED
667399	RealPlayer G2 LiveConnect plugin crash	--	RESOLVED
686782	GFX glitches on windows 8 (with D2D)	--	RESOLVED
760417	When selecting "Pop out" for HTML5 videos, the window is opened in full screen mode and the "Full screen" button is still available on its right bottom side	--	RESOLVED
778819	Toggle Fullscreen in context menu stops workng Flash 11.3.300.268	--	RESOLVED
785754	Buttons on some sites are not rendered right and are like half cut off	--	VERIFIED
786655	On flash content the context menu does not own focus	--	RESOLVED
786970	Windowed plugin content gets displayed out of it's container when scrolling the browser window	--	RESOLVED
786972	Can't replay the same HTML5 youtube video by clicking on the progress bar after when it ends	--	RESOLVED
787056	MapsGL maps.google.com renders blurry on machines Chrome doesn't allow MapsGL enabling on	P3	REOPENED
788423	[adbe 3612667] Flash hang in F_1152915508 on dropbox.com	--	RESOLVED
797116	[Win8] Flash hang in F_1152915508 while playing Knight Hunters	P3	RESOLVED
798295	On Flash content - copy actions are sometimes unresponsive	--	RESOLVED
800928	Context menu for a dragged vimeo tab in a new window is not shown correctly	P4	RESOLVED
800940	On Imdb, the video flash player of a dragged tab in a new window is not in the right position	P3	RESOLVED

16 Total; 1 Open (6.25%); 14 Resolved (87.5%); 1 Verified (6.25%);

@@ Line 958: / Line 958: @@
   |-
   | '''ID'''
-  |
+  | 20121031065642
   |-
   | '''Changeset'''
-  | [http://hg.mozilla.org/releases/mozilla-beta/rev/ ID] ([http://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=f8a6138bcfba&tochange=ID pushlog])
+  | [http://hg.mozilla.org/releases/mozilla-beta/rev/486335dcb4a2 486335dcb4a2] ([http://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=f8a6138bcfba&tochange=486335dcb4a2 pushlog])
   |}
@@ Line 1,061: / Line 1,061: @@
   | '''result''' - YYYY-MM-DD
   |}
+==== Additional Manual Spotchecks ====
+Due to automation infrastructure issues we will conduct the following manual spotchecks before signing off the Beta.
+{|class="querytable"
+ |class="header" | Test Area
+ |class="header" | Windows 7 64-bit
+ |class="header" | Windows 8 32-bit
+ |class="header" | Windows 8 64-bit
+ |-
+ | Bookmarks
+ | '''result''' (tester)
+ | '''result''' (tester)
+ | '''result''' (tester)
+ |-
+ | Private Browsing
+ | '''result''' (tester)
+ | '''result''' (tester)
+ | '''result''' (tester)
+ |-
+ | Session Restore
+ | '''result''' (tester)
+ | '''result''' (tester)
+ | '''result''' (tester)
+ |-
+ | Tabbed Browsing
+ | '''result''' (tester)
+ | '''result''' (tester)
+ | '''result''' (tester)
+ |-
+ | Add-ons & Themes
+ | '''result''' (tester)
+ | '''result''' (tester)
+ | '''result''' (tester)
+ |}
+; Test Area Details
+* Bookmarks
+** Add, Remove, Reorder, Tag, Keyword, Live (RSS)
+* Private Browsing
+** Authenticated session, cookies, cache, history
+* Session Restore
+** Normal restart, Crash, Multiple crash
+* Tabbed Browsing
+** Pinned tabs, Panorama, Tab scrolling, Tab reordering, Multi-window, Multi-monitor
+* Add-ons & Themes
+** Install several add-ons and themes from AMO and third-party, verify their functionality
 === Bug Fix Verifications ===

Releases/Firefox 17/Test Plan: Difference between revisions

Revision as of 22:49, 31 October 2012

Firefox 17 Test Plan

Summary

Details

Nightly

Merge

Features

Features

Endurance Automation

Bugzilla query error

Functional Automation

Bug Triage

Aurora Merge

Automated Testing

Manual Testing

Plug-in Compatibility

Web Compatibility

OSX 10.5 EOL

Feature Sign-offs

Update Testing

Bug Triage

Beta 1

Automated Testing

Manual Testing

Update Testing

Feature Sign-off

Build #2 Verifications

Beta 2

Automated Testing

Manual Testing

Update Testing

Feature Sign-off

Beta 3

Automated Testing

Manual Testing

Bug Fix Verifications

Update Testing

Beta 4

Automated Testing

Manual Testing

Additional Manual Spotchecks

Bug Fix Verifications

Update Testing

Bugs

Triage

New Issues

Known Issues

Navigation menu

Search