Revision as of 22:53, 28 November 2012

« Firefox 16 | Firefox 17 | Firefox 18 »

Firefox 17 Test Plan

Summary

Milestone	Firefox 17
Status	Currently in Beta -- merging to Released on November 20, 2012
Lead	Anthony Hughes (irc: ashughes)
2nd	Juan Becerra (irc: juanb)
Builds	[ftp] [http] [web]

The following is the test plan for Firefox 17 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 17. After the release this document will be used as an archive of what was done to validate this release.

If you have some free time, please pick a task below and get in touch with one of the leads.

Details

The following is a description of the tasks which need to be completed throughout this release.

2012-07-16: [DONE] Nightly Merge
2012-08-27: [DONE] Aurora Merge
2012-10-08: [DONE] Pre-Beta feature sign-offs
2012-10-11: [DONE] Beta 1
2012-10-18: {{|}} Beta 2
2012-10-25: {{|}} Beta 3
2012-11-01: {{|}} Beta 4
2012-11-08: {{|}} Beta 5
2012-11-15: {{|}} Beta 6
2012-11-19: {{|}} Release Candidate
2012-11-20: {{|}} Final

Nightly

Merge

Functional Automation - result

Update Spotchecks - result

Features

See the Feature Release Tracking wiki for a general overview

Features

See the Feature Release Tracking wiki for a general overview

Feature	Owner	Status	Tests
Improve display of location bar results	Virgil Dicu	On track
Firefox Social Integration	Anthony Hughes	On track
HTML Tree Editor	Mihaela Velimiroviciu
Silent Update not now prompt	Ioana Budnar	Release target removed.

Endurance Automation

The purpose of this section is to track any potential memory regressions via spikes reported in the dashboard.

What to Do

Check the Charts
If you see an anomalous event (ie. spike or dip), get the report for the specific event
If the event happens twice in a row, file a bug under Mozmill Tests
Notify Anthony Hughes of the issue
Record the event in this wiki below

Current Status (Mihaela Velimiroviciu) - Last Updated

Bugzilla query error

Query options must be valid JSON.1

Functional Automation

The purpose of this section is to track any potential functional regressions via results reported in the dashboard.

What to Do

Check the testrun reports
If you see any failing tests, check that there are not any bugs on file for it already
If not, file a bug under Mozmill Tests
Notify Anthony Hughes of the issue
Record the event in this wiki below

Current Status (Vlad Maniac / Alex Lakatos) - Last Updated

Bug Triage

The purpose of this section is to triage UNCONFIRMED incoming bugs.

What to Do

Check the UNCONFIRMED bugs
Test the bugs in the latest Firefox Nightly
If you can't reproduce the bug, ask for the reporter to provide more information
If you reproduce the bug, add your testing information and mark the bug NEW
Notify Anthony Hughes of the NEW issue

Current Status (SoftVision) - Last Updated

Result Spreadsheet

Aurora Merge

NOTE - OSX 10.5 should not be able to install nor update to Firefox 17


Builds	nightly
Changeset	tip (pushlog)

Checklist

[DONE] Automated Testing
[DONE] Manual Testing
[DONE] Update Testing
[DONE] Bug Triage
[DONE] Peer Sign-off
[ON TRACK] Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	PASS: report
Windows XP	PASS - 2012-08-28
Windows Vista	PASS - 2012-08-28
Windows 7 32-bit	PASS - 2012-08-28
Windows 7 64-bit	PASS - 2012-08-28
Windows 8 32-bit	N/A (pending IT)
Windows 8 64-bit	N/A (pending IT)
Mac OS X 10.6	PASS - 2012-08-28
Mac OS X 10.7	PASS - 2012-08-28
Mac OS X 10.8	PASS - 2012-08-28
Ubuntu 12.04 32-bit	PASS - 2012-08-28
Ubuntu 12.04 64-bit	PASS - 2012-08-28

Test Failures (known bugs)

Manual Testing

Operating System	Tester	Plug-in Compatibility	Web Compatibility	OSX 10.5 EOL
Windows XP	Simona	PASS - 2012-08-29	PASS - 2012-08-29	N/A
Windows 7	Ioana	PASS - 2012-08-29	PASS - 2012-08-29	N/A
Windows 8	Paul	PASS - 2012-08-30	PASS - 2012-08-30	N/A
Mac OS X	VladG	PASS - 2012-08-30	PASS - 2012-08-30	PASS - 2012-08-30
Ubuntu 12.04	Ioana	PASS - 2012-08-30	PASS - 2012-08-30	N/A

Plug-in Compatibility

Install the following plug-ins and go to websites which use them, looking for rendering issues, functional issues, performance issues, and crashes.

Adobe Flash Player
- 11.4 Release (make sure you can autoupdate)
Microsoft Silverlight
RealPlayer
QuickTime

Web Compatibility

Test the following websites for rendering issues, functional issues, performance issues, and crashes

Overall Top 10
Top by Country (pick a random site from three random countries)
Top by Category (pick a random site from three random categories)
Webmail clients like gmail, hotmail, and yahoo mail (pay particular attention to any built in web messenger software)
Try Google's MapsGL demo
Try YouTube's HTML5 demo

OSX 10.5 EOL

Verify the following:

Mac OSX 10.5 cannot download Firefox 17 (I'm not sure if this is scoped but try it anyway)
Mac OSX 10.5 cannot start Firefox 17
Mac OSX 10.5 cannot update to Firefox 17

The build can be downloaded and installed
Aurora didn't start, showing a incompatibility message
The update channel seems to be close so I didn't manage to perform an update

Feature Sign-offs

Log in to moztrap.mozilla.org
Select Run Tests
Select Product: Desktop Firefox
Select Versions: 17
Select Runs: Firefox 17 Aurora Merge Feature sign off

Feature	Owner	Win32	Mac	Linux
Improve display of location bar results	Virgil Dicu	PASS	PASS	PASS
Firefox Social Integration	Anthony Hughes	PASS	PASS	PASS

Details

Report any detailed results or bugs here...

Update Testing

Spotcheck updates from a previous Aurora to Aurora 17.

Platform	Auroratest	Aurora
Windows 7 64-bit	PASS - 2012-08-30	PASS - 2012-08-30
Windows 8 64-bit	PASS - 2012-08-30	PASS - 2012-08-30
Mac OS X 10.7	PASS - 2012-08-30	PASS - 2012-08-30
Ubuntu 12.04 64-bit	PASS - 2012-08-30	PASS - 2012-08-30

Issues

Bug Triage

VERIFYME (5): verify the fix and do some exploratory testing around the issue
FIXED (139): triage these bugs and test to ensure the fix is effective
QAWANTED (6): triage these bugs and test to fulfill the QA request
UNCONFIRMED (8): triage these bugs for potential new regressions

Results

VERIFIED: 24/139
WONTVERIFY: 3/139
HELPWANTED: 9/139

Beta 1

Builds	~~#1: FTP, HTTP~~ #2: FTP, HTTP
ID	~~#1: 20121009144933~~ #2: 20121010150351
Changeset	6b83222781e3 (pushlog)

Advisory

RelEng issues in generating the builds (see bug 796959 comment 5)

Checklist

[DONE] Automated Testing
[DONE] Manual Testing
[DONE] Update Testing
[DONE] Bug Triage
[DONE] Peer Sign-off
[DONE] Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	Build #1 Results (report)	Build #2 Results (report)
Windows XP	PASS - 2012-10-09	PASS - 2012-10-10
Windows Vista	PASS - 2012-10-09	PASS - 2012-10-10
Windows 7 32-bit	PASS - 2012-10-09	PASS - 2012-10-10
Windows 7 64-bit	PASS - 2012-10-09	PASS - 2012-10-10
Windows 8 32-bit	PASS - 2012-10-09	PASS - 2012-10-10
Windows 8 64-bit	PASS - 2012-10-09	PASS - 2012-10-10
Mac OS X 10.6	PASS - 2012-10-09	PASS - 2012-10-10
Mac OS X 10.7	PASS - 2012-10-09	PASS - 2012-10-10
Mac OS X 10.8	PASS - 2012-10-09	PASS - 2012-10-10
Ubuntu 12.04 32-bit	PASS - 2012-10-09	PASS - 2012-10-10
Ubuntu 12.04 64-bit	PASS - 2012-10-09	PASS - 2012-10-10

Test Failures (known bugs)

Manual Testing

Run the Desktop Firefox Regression Tests in MozTrap and report the results below. Do some exploratory testing around the testcases to try to exercise some edge-cases.

Platform	Tester	Result
Windows 7	Manuela, Ioana	PASS - 2012-10-11
Windows 8	Virgil	PASS - 2012-10-11
Mac OSX 10.7	Ioana, Mihaela	PASS - 2012-10-11
Mac OSX 10.8	Virgil	PASS - 2012-10-11
Ubuntu 12.04	Manuela	PASS - 2012-10-11
Special Cases
Mac OSX 10.5 de-support	Virgil	PASS - 2012-10-11
Update regression tests (requires updates on releasetest)	ashughes	PASS - 2012-10-11

Update Testing

Partial: 16.0b6, 16.0b5
Complete: <= 16.0b4

Platform	Releasetest (report)	Beta (report)
Windows XP	PASS - 2012-10-11	PASS - 2012-10-11
Windows Vista	PASS - 2012-10-11	PASS - 2012-10-11
Windows 7 32-bit	PASS - 2012-10-11	PASS - 2012-10-11
Windows 7 64-bit	PASS - 2012-10-11	PASS - 2012-10-11
Windows 8 32-bit	PASS - 2012-10-11	PASS - 2012-10-11
Windows 8 64-bit	PASS - 2012-10-11	PASS - 2012-10-11
Mac OS X 10.6	PASS - 2012-10-11	PASS - 2012-10-11
Mac OS X 10.7	PASS - 2012-10-11	PASS - 2012-10-11
Mac OS X 10.8	PASS - 2012-10-11	PASS - 2012-10-11
Ubuntu 12.04 32-bit	PASS - 2012-10-11	PASS - 2012-10-11
Ubuntu 12.04 64-bit	PASS - 2012-10-11	PASS - 2012-10-11

Issues

Feature Sign-off

Feature owners to test and sign-off overall quality of their features following uplift to Beta.

Feature	Owner	Win32	Mac	Linux
Improve display of location bar results	Virgil Dicu	n/a release target should be updated	n/a release target should be updated	n/a release target should be updated
HTML tree editor	Mihaela Velimiroviciu	PASS - 2012-10-11	PASS - 2012-10-11	PASS - 2012-10-11
Firefox Social Integration	Anthony Hughes	n/a - delayed to Beta 2	n/a - delayed to Beta 2	n/a - delayed to Beta 2

Build #2 Verifications

VERIFIED FIXED bug 799952
VERIFIED FIXED bug 720619
RESOLVED FIXED bug 798045 (could not verify because could not be reproduced)

Beta 2

Builds	FTP, HTTP
ID	20121017073013
Changeset	2ba4becf6e35 (pushlog)

Checklist

[DONE] Automated Testing
[DONE] Manual Testing
[ON TRACK] Update Testing
[ON TRACK] Bug Triage
[ON TRACK] Peer Sign-off
[ON TRACK] Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	Results (report)
Windows XP	PASS - 2012-10-17
Windows Vista	PASS - 2012-10-17
Windows 7 32-bit	PASS - 2012-10-17
Windows 7 64-bit	PASS - 2012-10-17
Windows 8 32-bit	PASS - 2012-10-17
Windows 8 64-bit	PASS - 2012-10-17
Mac OS X 10.6	PASS - 2012-10-17
Mac OS X 10.7	PASS - 2012-10-17
Mac OS X 10.8	PASS - 2012-10-17
Ubuntu 12.04 32-bit	PASS - 2012-10-17
Ubuntu 12.04 64-bit	PASS - 2012-10-17

Test Failures (known bugs)

testPrivateBrowsing/testDownloadManagerClosed.js (known)

Manual Testing

Run the Firefox 17b2 Regression Tests in MozTrap and report the results below. Do some exploratory testing around the testcases to try to exercise some edge-cases.

Moztrap > Desktop Firefox > 17 > Firefox 17b2 Regression Tests > Operating System > Platform > Run Tests

Platform	Tester	Result (report)
Windows 7	Manuela	PASS - 2012-10-18
Windows 8	Simona	PASS - 2012-10-18
Mac OSX 10.7	Paul	PASS - 2012-10-18
Mac OSX 10.8	Mihaela	PASS - 2012-10-18
Ubuntu 12.04	Virgil	PASS - 2012-10-18
Special Cases
Mac OSX 10.5 de-support	Virgil	PASS - 2012-10-18
Update regression tests (requires updates on releasetest)	ashughes	PASS - 2012-10-18

Update Testing

Partial: 17.0b1, 16.0b6
Complete: <=16.0b5

Platform	Releasetest (report)	Beta ([ report])
Windows XP	PASS - 2012-10-18	result - YYYY-MM-DD
Windows Vista	PASS - 2012-10-18	result - YYYY-MM-DD
Windows 7 32-bit	PASS - 2012-10-18	result - YYYY-MM-DD
Windows 7 64-bit	PASS - 2012-10-18	result - YYYY-MM-DD
Windows 8 32-bit	PASS - 2012-10-18	result - YYYY-MM-DD
Windows 8 64-bit	PASS - 2012-10-18	result - YYYY-MM-DD
Mac OS X 10.6	PASS - 2012-10-18	result - YYYY-MM-DD
Mac OS X 10.7	PASS - 2012-10-18	result - YYYY-MM-DD
Mac OS X 10.8	PASS - 2012-10-18	result - YYYY-MM-DD
Ubuntu 12.04 32-bit	PASS - 2012-10-18	result - YYYY-MM-DD
Ubuntu 12.04 64-bit	PASS - 2012-10-18	result - YYYY-MM-DD

Manual Spotchecks (due to Jenkins outage)

Update	Windows 7	Windows 8	Mac OSX 10.7	Mac OSX 10.8	Ubuntu 12.04
17.0b1 -> 17.0b2 en-US	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22
17.0b1 -> 17.0b2 de	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22
16.0b6 -> 17.0b2 en-US	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22
16.0b6 -> 17.0b2 zh-TW	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22
16.0b1 -> 17.0b2 en-US	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22
16.0b1 -> 17.0b2 es-ES	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22
15.0b6 -> 17.0b2 en-US	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22
15.0b6 -> 17.0b2 pt-BR	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22	PASS - 2012-10-22

Issues

Feature Sign-off

Feature owners to test and sign-off overall quality of their features following uplift to Beta.

Feature	Owner	Win32	Mac	Linux
Firefox Social Integration	Anthony Hughes	PASS - 2012-10-18	PASS - 2012-10-18	PASS - 2012-10-18

Issues

Server-side issues not blocking 17b2 (should be fixed with next server-side push)
- Like button activity not pushing to website
- Full page loading within panel (friends and notifications button)

Beta 3

Builds	FTP, HTTP
ID	20121023124120
Changeset	f8a6138bcfba (pushlog)

Checklist

[DONE] Automated Testing
[DONE] Manual Testing
[DONE] Update Testing
[DONE] Bug Triage
[DONE] Peer Sign-off
[DONE] Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	Results (report)
Windows XP	PASS - 2012-10-24
Windows Vista	PASS - 2012-10-24
Windows 7 32-bit	PASS - 2012-10-24
Windows 7 64-bit	PASS - 2012-10-24
Windows 8 32-bit	PASS - 2012-10-24
Windows 8 64-bit	PASS - 2012-10-24
Mac OS X 10.6	PASS - 2012-10-24
Mac OS X 10.7	PASS - 2012-10-24
Mac OS X 10.8	PASS - 2012-10-24
Ubuntu 12.04 32-bit	PASS - 2012-10-24
Ubuntu 12.04 64-bit	PASS - 2012-10-24

Test Failures (known bugs)

testPrivateBrowsing/testDownloadManagerClosed.js (known)

Manual Testing

Run the Firefox 17b3 Regression Tests in MozTrap and report the results below. Do some exploratory testing around the testcases to try to exercise some edge-cases.

Moztrap > Desktop Firefox > 17 > Firefox 17b3 Regression Tests > Operating System > Platform > Run Tests

Platform	Tester	Result (report)
Windows 7	Simona	PASS - 2012-10-24
Windows 8	Manuela	PASS - 2012-10-24
Mac OSX 10.7	Virgil	PASS - 2012-10-24
Mac OSX 10.8	Mihaela	PASS - 2012-10-24
Ubuntu 12.04	Ioana	PASS - 2012-10-24
Special Cases
Mac OSX 10.5 de-support	Virgil	PASS - 2012-10-24
Update regression tests (requires updates on releasetest)	ashughes	result - YYYY-MM-DD

Bug Fix Verifications

bug 793969 - verified (mwobensmith)
bug 801305 - verified (mwobensmith)
bug 802557 - verified (mwobensmith)
bug 803583 - delayed to 17b4 (juanb)
bug 803181 - verified (virgil)

Update Testing

Partial: 17.0b2, 17.0b1
Complete: <=16.0b6

Platform	Releasetest (report)	Beta ([ report])
Windows XP	PASS - 2012-10-24	result - YYYY-MM-DD
Windows Vista	PASS - 2012-10-24	result - YYYY-MM-DD
Windows 7 32-bit	PASS - 2012-10-24	result - YYYY-MM-DD
Windows 7 64-bit	PASS - 2012-10-24	result - YYYY-MM-DD
Windows 8 32-bit	PASS - 2012-10-24	result - YYYY-MM-DD
Windows 8 64-bit	PASS - 2012-10-24	result - YYYY-MM-DD
Mac OS X 10.6	PASS - 2012-10-24	result - YYYY-MM-DD
Mac OS X 10.7	PASS - 2012-10-24	result - YYYY-MM-DD
Mac OS X 10.8	PASS - 2012-10-24	result - YYYY-MM-DD
Ubuntu 12.04 32-bit	PASS - 2012-10-24	result - YYYY-MM-DD
Ubuntu 12.04 64-bit	PASS - 2012-10-24	result - YYYY-MM-DD

Issues

Beta 4

Builds	FTP, HTTP
ID	20121031065642
Changeset	486335dcb4a2 (pushlog)

Checklist

[DONE] Automated Testing
[DONE] Manual Testing
[DONE] Update Testing
[ON TRACK] Bug Triage
[DONE] Peer Sign-off
[DONE] Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	Results (report)
Windows XP	PASS - 2012-11-01
Windows Vista	PASS - 2012-11-01
Windows 7 32-bit	PASS - 2012-11-01
Windows 7 64-bit	PASS - 2012-11-01
Windows 8 32-bit	PASS - 2012-11-01
Windows 8 64-bit	PASS - 2012-11-01
Mac OS X 10.6	PASS - 2012-11-01
Mac OS X 10.7	PASS - 2012-11-01
Mac OS X 10.8	PASS - 2012-11-01
Ubuntu 12.04 32-bit	PASS - 2012-11-01
Ubuntu 12.04 64-bit	PASS - 2012-11-01

Test Failures (known bugs)

testPrivateBrowsing/testDownloadManagerClosed.js (known)

Manual Testing

Run the Firefox 17b4 Regression Tests in MozTrap and report the results below. Do some exploratory testing around the testcases to try to exercise some edge-cases.

Moztrap > Desktop Firefox > 17 > Firefox 17b4 Regression Tests > Operating System > Platform > Run Tests

If you have an AMD or NVIDIA graphics card, please install the Beta drivers during your testing

AMD Catalyst 12.11 Beta (download)
- AMD Catalyst 12.11 Beta graphics driver is not available for Mac OS X.
NVidia GeForce 310.33 Beta (download)
- NVidia GeForce 310.33 Beta graphics driver is not compatible with Windows 8.
- NVidia GeForce 310.33 Beta graphics driver is not available for Mac OS X.
keep your eye out for rendering issues and crashes

Platform	Tester	Result (report)
Windows 7	Paul	PASS - 2012-11-01
Windows 8	Simona	PASS - 2012-11-02
Mac OSX 10.7	Virgil	PASS - 2012-11-01
Mac OSX 10.8	Mihaela	PASS - 2012-11-01
Ubuntu 12.04	Ioana	PASS - 2012-11-02
Special Cases
Mac OSX 10.5 de-support	Virgil	PASS - 2012-11-01
Update regression tests (requires updates on releasetest)	ashughes	PASS - 2012-11-02

Bug Fix Verifications

bug 803583 - verified (juanb)
bug 776247 - verified (jsmith)
bug 806991 - n/a - to be tested after 17b4 is live (ashughes)

Update Testing

Partial: 17.0b3, 17.0b2
Complete: <=17.0b1

Platform	Releasetest (report)	Beta (report)
Windows XP	PASS - 2012-11-02	PASS - 2012-11-02
Windows Vista	PASS - 2012-11-02	PASS - 2012-11-02
Windows 7 32-bit	PASS - 2012-11-02	PASS - 2012-11-02
Windows 7 64-bit	PASS - 2012-11-02	PASS - 2012-11-02
Windows 8 32-bit	PASS - 2012-11-02	PASS - 2012-11-02
Windows 8 64-bit	PASS - 2012-11-02	PASS - 2012-11-02
Mac OS X 10.6	PASS - 2012-11-02	PASS - 2012-11-02
Mac OS X 10.7	PASS - 2012-11-02	PASS - 2012-11-02
Mac OS X 10.8	PASS - 2012-11-02	PASS - 2012-11-02
Ubuntu 12.04 32-bit	PASS - 2012-11-02	PASS - 2012-11-02
Ubuntu 12.04 64-bit	PASS - 2012-11-02	PASS - 2012-11-02

Issues

Beta 5

Builds	FTP, HTTP
ID	20121106195758
Changeset	036da70b61f9 (pushlog)

Checklist

[DONE] Automated Testing
[DONE] Manual Testing
[DONE] Update Testing
[ON TRACK] Bug Triage
[DONE] Peer Sign-off
[DONE] Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	Results (report)
Windows XP	PASS - 2012-11-06
Windows Vista	PASS - 2012-11-06
Windows 7 32-bit	PASS - 2012-11-06
Windows 7 64-bit	PASS - 2012-11-06
Windows 8 32-bit	PASS - 2012-11-06
Windows 8 64-bit	PASS - 2012-11-06
Mac OS X 10.6	PASS - 2012-11-06
Mac OS X 10.7	PASS - 2012-11-06
Mac OS X 10.8	PASS - 2012-11-06
Ubuntu 12.04 32-bit	PASS - 2012-11-06
Ubuntu 12.04 64-bit	PASS - 2012-11-06

Test Failures (known bugs)

Manual Testing

Special Instructions

While testing this Beta, please install the latest AMD or NVidia beta drivers (if you have a GPU) and Flash 11.5 (if you are on Windows). Report any visual problems, crashes, or hangs in Bugzilla and add them to the New Issues section below.

Regression Tests

Run the Firefox 17b5 Regression Tests in MozTrap and report the results below. Do some exploratory testing around the testcases to try to exercise some edge-cases.

Moztrap > Desktop Firefox > 17 > Firefox 17b5 Regression Tests > Operating System > Platform > Run Tests

Platform	Tester	Result (report)	Bugs
Windows 7	Manuela	PASS - 2012-11-07
Windows 8	Simona	PASS - 2012-11-07
Mac OSX 10.8	Virgil	PASS - 2012-11-07
Ubuntu 12.04	Mihaela	PASS - 2012-11-07
Mac OSX 10.5 de-support	Paul	PASS - 2012-11-07
Update regression tests	ashughes	PASS - 2012-11-07

Social API

In addition to normal dogfooding (sidebar, activity stream, chat, toolbar), please heavily focus on testing around Private Browsing. In this Beta, the Social API should be disabled in Private Browsing mode. Test to make sure Social API cannot be enabled in Private Browsing, your Social API session is restored when exiting Private Browsing, and that Private Browsing mode works as per normal for all other use cases (Bookmarks, History, Cache, Logins, Cookies, etc)

Platform	Tester	Result	Bugs
Windows 7	Simona	PASS - 2012-11-08	807217#c43 807675#c3
Mac OSX 10.8	Virgil	PASS - 2012-11-08
Ubuntu 12.04	Ioana	PASS - 2012-11-08	807217#c45

Click-to-Play

Firefox 17 includes a click-to-play blocking feature which should show an overlay when plug-in content loads. Clicking the overlay should play the content. This feature has not been deployed for all plug-ins however. Flash <=11.3, Java 7 <=7u8, Java 6 <=6u36, Reader <=10.1.3, Silverlight <=5.1.10410.0 should all be CTP blocked. All the latest versions of the plug-ins should not be blocked.

Recent regressions indicate not all sites respect CTP blocking (CNN and LA Times for example). Please check various websites with plug-in content. If you can, work through the top-100 list and fill in that table with your results. However, don't be afraid to go outside that list if there are other sites you can think of.

Platform	Tester	Result	Bugs
Windows 7	Virgil, Manuela	FAIL - 2012-11-07	809792, 809858
Mac OSX 10.8	Manuela, Paul	FAIL - 2012-11-07	809792
Ubuntu 12.04	Ioana, Paul	FAIL - 2012-11-08	809785, 809793, 809860

Bug Fix Verifications

Reopened bug 788423 (ioanab)
Verified bug 796039 (psilaghi)
Verified bug 797163 (mwobensmith)
Pending bug 807217 (ashughes) -- Facebook outage, should be resolved by 5pm Wednesday

Update Testing

Partial: 17.0b4, 17.0b3
Complete: <=17.0b2

Platform	Releasetest (report)	Beta (report)
Windows XP	PASS - 2012-11-07	PASS - 2012-11-08
Windows Vista	PASS - 2012-11-07	PASS - 2012-11-08
Windows 7 32-bit	PASS - 2012-11-07	PASS - 2012-11-08
Windows 7 64-bit	PASS - 2012-11-07	PASS - 2012-11-08
Windows 8 32-bit	PASS - 2012-11-07	PASS - 2012-11-08
Windows 8 64-bit	PASS - 2012-11-07	PASS - 2012-11-08
Mac OS X 10.6	PASS - 2012-11-07	PASS - 2012-11-08
Mac OS X 10.7	PASS - 2012-11-07	PASS - 2012-11-08
Mac OS X 10.8	PASS - 2012-11-07	PASS - 2012-11-08
Ubuntu 12.04 32-bit	PASS - 2012-11-07	PASS - 2012-11-08
Ubuntu 12.04 64-bit	PASS - 2012-11-07	PASS - 2012-11-08

Issues

Beta 6

Builds	FTP, HTTP
ID	20121113065533
Changeset	0dbbf9b3175c (pushlog)

Checklist

[DONE] Automated Testing
[DONE] Manual Testing
[DONE] Update Testing
[DONE] Bug Triage
[DONE] Peer Sign-off
{{|}} Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	Results (report)
Windows XP	4 failures - 2012-11-12
Windows Vista	0 failures - 2012-11-12
Windows 7 32-bit	0 failures - 2012-11-12
Windows 7 64-bit	0 failures - 2012-11-12
Windows 8 32-bit	0 failures - 2012-11-12
Windows 8 64-bit	0 failures - 2012-11-12
Mac OS X 10.6	0 failures - 2012-11-12
Mac OS X 10.7	0 failures - 2012-11-12
Mac OS X 10.8	0 failures - 2012-11-12
Ubuntu 12.04 32-bit	0 failures - 2012-11-12
Ubuntu 12.04 64-bit	0 failures - 2012-11-12

Test Failures (known bugs)

bug 811407 Awesomebar tests failed for 17.0b6 candidate on Windows XP resulting in Disconnect error

Manual Testing

Special Instructions

While testing this Beta, please install the latest AMD or NVidia beta drivers (if you have a GPU) and Flash 11.5 (if you are on Windows). Report any visual problems, crashes, or hangs in Bugzilla and add them to the New Issues section below.

Regression Tests

Run the Firefox 17b6 Regression Tests in MozTrap and report the results below. Do some exploratory testing around the testcases to try to exercise some edge-cases.

Moztrap > Desktop Firefox > 17 > Firefox 17b6 Regression Tests > Operating System > Platform > Run Tests

Platform	Tester	Result (report)	Bugs
Windows 7	Virgil	PASS - 2012-11-14
Windows 8	Simona	PASS - 2012-11-14
Mac OSX 10.8	Ioana	PASS - 2012-11-14
Ubuntu 12.04	Manuela	PASS - 2012-11-14
Mac OSX 10.5 de-support	Mihaela	PASS - 2012-11-14
Update regression tests	ashughes	PASS - 2012-11-14

Awesomebar

Check the Awesomebar for potential regressions. In particular, bookmarking, searching, filtering, security information, and switch-to-tab.

Platform	Tester	Result	Bugs
Windows XP	Virgil	PASS - 2012-11-14
Windows 8	Simona	PASS - 2012-11-14
Mac OSX 10.8	Mihaela	PASS - 2012-11-14
Ubuntu 12.04	Manuela	PASS - 2012-11-14

Private Browsing

Test Private Browsing mode and check for any potential regressions. In particular history, session and cache leaking into the normal browsing session.

Platform	Tester	Result	Bugs
Windows XP	Paul	PASS - 2012-11-14
Windows 8	Virgil	PASS - 2012-11-14
Mac OSX 10.8	Mihaela	PASS - 2012-11-14
Ubuntu 12.04	Manuela	PASS - 2012-11-14

Bug Fix Verifications

Downloads Panel

verified bug 809022 (simonab)

Social API

verified bug 808116 (ashughes)

VERIFYME (bugs) - aiming to verify all blocker/critical/major bugs

verified bug 766616 (ioana)
verified bug 680644 (virgil)
verified bug 774250 (paul)

Update Testing

Partial: 17.0b5, 17.0b4
Complete: <=17.0b3

Platform	Releasetest (report)	Beta (report)
Windows XP	PASS - 2012-11-14	PASS - 2012-11-14
Windows Vista	PASS - 2012-11-14	PASS - 2012-11-14
Windows 7 32-bit	PASS - 2012-11-14	PASS - 2012-11-14
Windows 7 64-bit	PASS - 2012-11-14	PASS - 2012-11-14
Windows 8 32-bit	PASS - 2012-11-14	PASS - 2012-11-14
Windows 8 64-bit	PASS - 2012-11-14	PASS - 2012-11-14
Mac OS X 10.6	PASS - 2012-11-14	PASS - 2012-11-14
Mac OS X 10.7	PASS - 2012-11-14	PASS - 2012-11-14
Mac OS X 10.8	PASS - 2012-11-14	PASS - 2012-11-14
Ubuntu 12.04 32-bit	PASS - 2012-11-14	PASS - 2012-11-14
Ubuntu 12.04 64-bit	PASS - 2012-11-14	PASS - 2012-11-14

Issues

Release Candidate

Builds	~~FTP, HTTP~~ FTP, HTTP
ID	~~20121116115405~~ 20121119183901
Changeset	~~0af23f8db35d (pushlog)~~ 919435c6f654 (pushlog)

Checklist (respin for bug 813264)

[DONE] Automated Testing
[DONE] Manual Testing
[DONE] Update Testing on betatest
[DONE] Functional sign-off
{{|}} Update testing on releasetest
{{|}} Update testing on release

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	Build#1 Results (report)	Build#2 Results (report)
Windows XP	PASS - 2012-11-16	1 failure - 2012-11-19
Windows Vista	PASS - 2012-11-16	PASS - 2012-11-19
Windows 7 32-bit	PASS - 2012-11-16	PASS - 2012-11-19
Windows 7 64-bit	PASS - 2012-11-16	PASS - 2012-11-19
Windows 8 32-bit	PASS - 2012-11-16	PASS - 2012-11-19
Windows 8 64-bit	PASS - 2012-11-16	PASS - 2012-11-19
Mac OS X 10.6	1 failure - 2012-11-16	PASS - 2012-11-19
Mac OS X 10.7	1 failure - 2012-11-16	PASS - 2012-11-19
Mac OS X 10.8	2 failures - 2012-11-16	1 failure - 2012-11-19
Ubuntu 12.04 32-bit	1 failure - 2012-11-16	PASS - 2012-11-19
Ubuntu 12.04 64-bit	1 failure - 2012-11-16	PASS - 2012-11-19

Test Failures (known bugs)

Build 1
- /restartTests/testAddons_uninstallExtension/test4.js - Shutdown expected but none detected before end of test (bug 783484)
- /testPreferences/testPreferredLanguage.js -could not find element Link: Documenti (bug 812435)
Build 2
- /testAwesomeBar/testSuggestHistory.js (bug 813449)
- /restartTests/testAddons_uninstallExtension/test4.js (bug 783484)

Manual Testing

Regression Tests

Run the Firefox 17 Regression Tests in MozTrap and report the results below. Do some exploratory testing around the testcases to try to exercise some edge-cases.

Build 1 Results

Platform	Tester	Result (report)	Bugs
Windows 7	Manuela	PASS - 2012-11-19
Windows 8	Simona	PASS - 2012-11-19
Mac OSX 10.8	Paul	PASS - 2012-11-19
Ubuntu 12.04	Mihaela	PASS - 2012-11-19
Mac OSX 10.5 de-support	Ioana	PASS - 2012-11-19
Update regression tests		PASS - 2012-11-19

Build 2 Results

Conduct focused regression testing around third-party binary add-ons and exploratory testing around web compatibility. Yandex, AVG, Bandoo, and Skype are some add-ons to test, but try to test outside that scope and find some other third-party binary add-ons. In terms of websites, try some of the Alexa Top-sites but also try to visit sites you might normally use outside of that scope (banking, video, webmail, news, games, etc).

[FAIL] Windows 7 (Manuela, Ioana)
- Addons Tested:
  - Hotspot Shield - PASS
  - Bandoo for Firefox - PASS
  - Skype Click to Call - FAIL: this addon is present in Firefox, but it doesn't work at all (after being forced as compatible and enabled). No phone numbers are highlighted (although they were when opening the same profile/pages etc in Firefox 15.0.1).
  - Yandex Toolbar - PASS
- Websites Tested: facebook.com, amazon.com, twitter.com, qq.com, live.com, wikipedia.org, google.com, yahoo.com, baidu.com, youtube.com, miniclip.com, trailers.apple.com - PASS
- Scenarios Tested: tested the addons for basic functionality, tested the websites for functionality, rendering and performance issues (with the addons installed).
[PASS] Windows 8 (Simona)
- Addons Tested:
  - BitTorrentBar - Pass
  - Facebook Toolbar - Pass
  - IncrediMail MediaBar 2 - Pass
  - Yahoo! Toolbar - Pass
  - Yandex Community Toolbar - Pass
  - Bandoo for Firefox 5.1 - Pass
  - Skype Click to call - I could not install Skype for Windows 8 - FAIL
- Websites Tested:
- Scenarios Tested: tested the add-ons basic functionality and websites rendering issue having all the add-ons enabled
[FAIL] Mac OSX 10.8 (Paul)
- Addons Tested:
  - Skype Click to Call - N/A for Firefox on Mac
  - Bandoo - N/A on Mac
  - http://www.siteadvisor.com/download/ff_preinstall.html - Pass
  - http://bittorrentbar.ourtoolbar.com/ - Pass
  - http://pages.tvunetworks.com/downloads/player.html - Pass
  - Facebook Toolbar - Pass
  - Yahoo! Toolbar - Pass
  - http://downloadmytoolbar.com/4shared/LP/download.html - FAIL (firefox totally freezes)
  - http://incredimailmediabar2.ourtoolbar.com/ - FAIL (firefox totally freezes)
  - yandex http://ct57527.ourtoolbar.com/ - FAIL (firefox totally freezes)
  - http://hotspotshield.ourtoolbar.com/ - FAIL (firefox totally freezes)
- Websites Tested:
- Scenarios Tested:
[FAIL] Ubuntu 12.04 (Mihaela)
- Addons Tested:
  - 4shared.com Community Toolbar3.15.1.0 - unstable (sometimes new tabs opened from + button are not shown until a browser restart)
  - BitTorrentBar Community Toolbar3.15.1.0 - PASS
  - Facebook Toolbar1.8.2 - PASS
  - Hotspot Shield Community Toolbar3.15.1.0 - breaks session restore (restore previous session button not displayed in about:home page, nor under History menu)
  - IncrediMail MediaBar 2 Community Toolbar3.15.1.0 - unstable (sometimes new tabs opened from + button are not shown until a browser restart)
  - McAfee SiteAdvisor28.0 - PASS
  - Yahoo! Toolbar2.5.1.20121012015120 - PASS
  - Yandex Community Toolbar3.15.1.0 - unstable (sometimes new tabs opened from + button are not shown until a browser restart), breaks session restore (restore previous session button not displayed in about:home page, nor under History menu)
- Websites Tested: google.com, facebook.com, www.youtube.com (flash and html), maps.google.com, mail.yahoo.com, yahoo.com
- Scenarios Tested: install/enable/disable add-on, enable/disable toolbar, add-ons basic functionality, websites basic functionality, pinned tabs
Issues Found:

Additional Spotchecks

[PASS] Mac OSX 10.5 Desupport (Virgil)

Bug Fix Verifications

VERIFYME (bugs) - aiming to verify all blocker/critical/major bugs

QAWANTED

bug 675260 crash in DefaultFreeEntry coming from ShutdownNSS (ashughes)
bug 770238 Fatal jemalloc assertion in js::gc::Arena::finalize<JSString> (marcia)
bug 812307 crash in DataMngrHlpFF17.dll @ nsVoidArray::~nsVoidArray (juanb)
bug 812456 Extension block request: DataMngr (virgil)
bug 812683 Win7 startup crash in d3d11.dll (after installing IE10?) (ioana)

Update Testing

Partial: 16.0.2, 15.0.1, 14.0.1
Complete: 16.0

Platform	betatest (report)	release (report)
Windows XP	PASS - 2012-11-20	PASS - 2012-11-20
Windows Vista	PASS - 2012-11-20	PASS - 2012-11-20
Windows 7 32-bit	PASS - 2012-11-20	PASS - 2012-11-20
Windows 7 64-bit	PASS - 2012-11-20	PASS - 2012-11-20
Windows 8 32-bit	FAIL - 2012-11-20	PASS - 2012-11-20
Windows 8 64-bit	FAIL - 2012-11-20	PASS - 2012-11-20
Mac OS X 10.6	PASS - 2012-11-20	PASS - 2012-11-20
Mac OS X 10.7	PASS - 2012-11-20	PASS - 2012-11-20
Mac OS X 10.8	PASS - 2012-11-20	PASS - 2012-11-20
Ubuntu 12.04 32-bit	PASS - 2012-11-20	PASS - 2012-11-20
Ubuntu 12.04 64-bit	PASS - 2012-11-20	PASS - 2012-11-20

Issues

Windows 8 VMs failed with "cannot connect to jsbridge extension, port 24242"
- Manual spotchecks succeeded

17.0.1

Builds	FTP, HTTP
ID	id
Changeset	id (pushlog)

Checklist (respin for bug 813264)

{{|}} Automated Testing
{{|}} Manual Testing
{{|}} Update Testing on betatest
{{|}} Functional sign-off
{{|}} Update Testing on release

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	Build#1 Results ([ report])
Windows XP	result - YYYY-MM-DD
Windows Vista	result - YYYY-MM-DD
Windows 7 32-bit	result - YYYY-MM-DD
Windows 7 64-bit	result - YYYY-MM-DD
Windows 8 32-bit	result - YYYY-MM-DD
Windows 8 64-bit	result - YYYY-MM-DD
Mac OS X 10.6	result - YYYY-MM-DD
Mac OS X 10.7	result - YYYY-MM-DD
Mac OS X 10.8	result - YYYY-MM-DD
Ubuntu 12.04 32-bit	result - YYYY-MM-DD
Ubuntu 12.04 64-bit	result - YYYY-MM-DD

Test Failures (known bugs)

Manual Testing

Regression Tests

Run the Firefox 17 Regression Tests in MozTrap and report the results below. Do some exploratory testing around the testcases to try to exercise some edge-cases.

Build 1 Results

Platform	Tester	Result (report)	Bugs
Windows XP	tester	result - YYYY-MM-DD
Windows 8	tester	result - YYYY-MM-DD
Mac OSX 10.8	tester	result - YYYY-MM-DD
Ubuntu 12.04	tester	result - YYYY-MM-DD
Mac OSX 10.5 de-support	tester	result - YYYY-MM-DD
Update regression tests	tester	result - YYYY-MM-DD

Additional Spotchecks

[result] Mac OSX 10.5 Desupport (tester)

Bug Fix Verifications

bug 806192
bug 815042
bug 815743
bug 814101
bug 813264
bug 814554
bug 816197

Update Testing

Partial:
Complete:

Platform	betatest ([ report])	release ([ report])
Windows XP	result - YYYY-MM-DD	result - YYYY-MM-DD
Windows Vista	result - YYYY-MM-DD	result - YYYY-MM-DD
Windows 7 32-bit	result - YYYY-MM-DD	result - YYYY-MM-DD
Windows 7 64-bit	result - YYYY-MM-DD	result - YYYY-MM-DD
Windows 8 32-bit	result - YYYY-MM-DD	result - YYYY-MM-DD
Windows 8 64-bit	result - YYYY-MM-DD	result - YYYY-MM-DD
Mac OS X 10.6	result - YYYY-MM-DD	result - YYYY-MM-DD
Mac OS X 10.7	result - YYYY-MM-DD	result - YYYY-MM-DD
Mac OS X 10.8	result - YYYY-MM-DD	result - YYYY-MM-DD
Ubuntu 12.04 32-bit	result - YYYY-MM-DD	result - YYYY-MM-DD
Ubuntu 12.04 64-bit	result - YYYY-MM-DD	result - YYYY-MM-DD

Issues

Bugs

Triage

VERIFYME: verify the fix and do some exploratory testing around the issue
FIXED: triage these bugs and test to ensure the fix is effective
QAWANTED: triage these bugs and test to fulfill the QA request
UNCONFIRMED: triage these bugs for potential new regressions

New Issues

Anything in bold is a potential blocker.

bug 813531 An unknown error while printing SVG file
bug 813553 [Mac OS] Firefox freezes after installing add-ons which launch a pop-up after installation
bug 813563 Cannot restore previous session (with Yandex Community Toolbar3.15.1.0 and Hotspot Shield Community Toolbar3.15.1.0)
bug 813566 [Mac OS] Mouse cursor disappears when switching tabs
bug 813568 Skype Click to Call doesn't work on Firefox 17, although it is enabled
bug 813572 New tab is not added to the tabs strip (with 4shared.com Community Toolbar, IncrediMail MediaBar 2 Community Toolbar)
bug 813594 Gaping tabs strip after installing Facebook Toolbar

Known Issues

Anything in bold is a potential blocker.

Click-to-Play

bug 809785 Plugin content can't be activated on sohu.com because it is covered by an <a> element
bug 809793 The Click-to-Play message is cropped on taobao.com
bug 809858 Click to play overlay not working on paypal.com

Graphics

bug 686782 GFX glitches on Windows 8 with D2D
bug 772837 Content of drop-down lists (awesomebar history) blink on secondary monitor with HWA

HTML5 Video

bug 760417 Full-screen button visible in pop-out full-screen mode
bug 786972 Can't replay video by clicking on the progress bar after playback ends
bug 803057 HTML5 videos continues to play after Pop Out
bug 708557 Html5 youtube context menu not persistent when right clicking

Layout

FIXED bug 785754 Buttons on some sites are not rendered correctly
FIXED bug 803074 Rendering is broken after restoring skew() support

Plugins

bug 667399 RealPlayer G2 LiveConnect plugin crash
WFM bug 786655 Flash context menu does not own focus
bug 786970 Windowed plugin content gets displayed out of it's container when scrolling the browser window
FIXED? bug 788423 [adbe 3325067] Flash hang in F_1152915508 on dropbox.com (test Nov 7, 2012)
bug 797116 Win8 Flash hang in F_1152915508 while playing Knight Hunters
WFM bug 798295 Copy actions are sometimes unresponsive in Flash
bug 800928 Incorrect display of Vimeo context menu in a tab dragged to a new window
bug 800940 Incorrect positioning of IMDB video player in a tab dragged to a new window
bug 797830 [flash 11.5] Right clicking the Flash content pauses the video when using the 11.5 beta version

Printing

bug 638796 Non-functional print button in Firefox Menu when viewing a PDF file

Social API

bug 789707 Make sure we're handling offline mode
FIXED bug 799856 crash in mozilla::InternalInvalidateThebesLayersInSubtree, canceled login to Facebook
bug 804622 Sidebar asks me to log in even though I'm logged in
bug 807673 Social chat panels are not displayed right when resizing Firefox vertically
bug 807675 Friends in the Social sidebar are sometimes marked as offline without actually going offline
bug 807915 Can't activate Social API from the Facebook activation page
bug 807935 The chat window persists in DOM full-screen mode
FIXED? bug 807981 Facebook new friend request notification link is not opened in new tab (test Nov 14)
bug 807997 Nothing happens when attempting to open a contact in Facebook sidebar with about: page open
bug 808001 Clicking on notification pop-ups opens them in the focused tab, not in a new tab
bug 811727 Can't turn on facebook messenger from facebook notification

Switch-to-Tab

bug 811668 "Switch to tab" switches to other tab group even if the same page is opened in current group

WebGL

bug 787056 Google MapsGL blurry on machines where Chrome MapsGL isn't supported

Toolbars

bug 518089 A hidden state of the toolbar which user added does not persist after reboot, When a multi-byte letter is included in a name of the toolbar

@@ Line 1,844: / Line 1,844: @@
 * Windows 8 VMs failed with "cannot connect to jsbridge extension, port 24242"
 ** Manual spotchecks succeeded
+== 17.0.1 ==
+{| class="querytable" style="width:50%"
+ | style="width:10%" | '''Builds'''
+ | [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/17.0.1-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/17.0.1-candidates/build1/ HTTP]
+ |-
+ | '''ID'''
+ | id
+ |-
+ | '''Changeset'''
+ | [http://hg.mozilla.org/releases/mozilla-release/rev/id id] ([http://hg.mozilla.org/releases/mozilla-release/pushloghtml?fromchange=919435c6f654&tochange=id pushlog])
+ |}
+; Checklist (respin for {{bug|813264}})
+* {{|}} [[Releases/Firefox_17/Test_Plan#Automated_Testing_9|Automated Testing]]
+* {{|}} [[Releases/Firefox_17/Test_Plan#Manual_Testing_9|Manual Testing]]
+* {{|}} [[Releases/Firefox_17/Test_Plan#Update_Testing_9|Update Testing]] on betatest
+* {{|}} Functional sign-off
+* {{|}} [[Releases/Firefox_17/Test_Plan#Update_Testing_9|Update Testing]] on release
+=== Automated Testing ===
+The following are the detailed results of the automated Mozmill test runs.
+{|class="querytable" style="width:60%"
+ |-
+ | class="header" style="width:20%" | Platform
+ | class="header" style="width:30%" | Build#1 Results ([ report])
+ |-
+ | Windows XP
+ | '''result''' - YYYY-MM-DD
+ |-
+ | Windows Vista
+ | '''result''' - YYYY-MM-DD
+ |-
+ | Windows 7 32-bit
+ | '''result''' - YYYY-MM-DD
+ |-
+ | Windows 7 64-bit
+ | '''result''' - YYYY-MM-DD
+ |-
+ | Windows 8 32-bit
+ | '''result''' - YYYY-MM-DD
+ |-
+ | Windows 8 64-bit
+ | '''result''' - YYYY-MM-DD
+ |-
+ | Mac OS X 10.6
+ | '''result''' - YYYY-MM-DD
+ |-
+ | Mac OS X 10.7
+ | '''result''' - YYYY-MM-DD
+ |-
+ | Mac OS X 10.8
+ | '''result''' - YYYY-MM-DD
+ |-
+ | Ubuntu 12.04 32-bit
+ | '''result''' - YYYY-MM-DD
+ |-
+ | Ubuntu 12.04 64-bit
+ | '''result''' - YYYY-MM-DD
+ |}
+; Test Failures ([https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA known bugs])
+=== Manual Testing ===
+==== Regression Tests ====
+Run the [https://moztrap.mozilla.org/runtests/environment/395/ Firefox 17 Regression Tests] in MozTrap and report the results below. Do some exploratory testing around the testcases to try to exercise some edge-cases.
+; Build 1 Results
+{|class="querytable"
+ |class="header" style="width:20%" | Platform
+ |class="header" style="width:20%" | Tester
+ |class="header" style="width:20%" | Result ([https://moztrap.mozilla.org/results/cases/?filter-run=395 report])
+ |class="header" style="width:40%" | Bugs
+ |-
+ | Windows XP
+ | tester
+ | '''result''' - YYYY-MM-DD
+ |
+ |-
+ | Windows 8
+ | tester
+ | '''result''' - YYYY-MM-DD
+ |
+ |-
+ | Mac OSX 10.8
+ | tester
+ | '''result''' - YYYY-MM-DD
+ |
+ |-
+ | Ubuntu 12.04
+ | tester
+ | '''result''' - YYYY-MM-DD
+ |
+ |-
+ | Mac OSX 10.5 de-support
+ | tester
+ | '''result''' - YYYY-MM-DD
+ |
+ |-
+ | Update regression tests
+ | tester
+ | '''result''' - YYYY-MM-DD
+ |
+ |}
+Additional Spotchecks
+* ['''result'''] Mac OSX 10.5 Desupport (tester)
+=== Bug Fix Verifications ===
+* bug 806192
+* bug 815042
+* bug 815743
+* bug 814101
+* bug 813264
+* bug 814554
+* bug 816197
+=== Update Testing ===
+* Partial:
+* Complete:
+{|class="querytable" style="width:60%"
+ |-
+ | class="header" | Platform
+ | class="header" | betatest ([ report])
+ | class="header" | release ([ report])
+ |-
+ | Windows XP
+ | '''result''' - YYYY-MM-DD
+ | '''result''' - YYYY-MM-DD
+ |-
+ | Windows Vista
+ | '''result''' - YYYY-MM-DD
+ | '''result''' - YYYY-MM-DD
+ |-
+ | Windows 7 32-bit
+ | '''result''' - YYYY-MM-DD
+ | '''result''' - YYYY-MM-DD
+ |-
+ | Windows 7 64-bit
+ | '''result''' - YYYY-MM-DD
+ | '''result''' - YYYY-MM-DD
+ |-
+ | Windows 8 32-bit
+ | '''result''' - YYYY-MM-DD
+ | '''result''' - YYYY-MM-DD
+ |-
+ | Windows 8 64-bit
+ | '''result''' - YYYY-MM-DD
+ | '''result''' - YYYY-MM-DD
+ |-
+ | Mac OS X 10.6
+ | '''result''' - YYYY-MM-DD
+ | '''result''' - YYYY-MM-DD
+ |-
+ | Mac OS X 10.7
+ | '''result''' - YYYY-MM-DD
+ | '''result''' - YYYY-MM-DD
+ |-
+ | Mac OS X 10.8
+ | '''result''' - YYYY-MM-DD
+ | '''result''' - YYYY-MM-DD
+ |-
+ | Ubuntu 12.04 32-bit
+ | '''result''' - YYYY-MM-DD
+ | '''result''' - YYYY-MM-DD
+ |-
+ | Ubuntu 12.04 64-bit
+ | '''result''' - YYYY-MM-DD
+ | '''result''' - YYYY-MM-DD
+ |}
+; Issues
 == Bugs ==

Releases/Firefox 17/Test Plan: Difference between revisions

Revision as of 22:53, 28 November 2012

Firefox 17 Test Plan

Summary

Details

Nightly

Merge

Features

Features

Endurance Automation

Bugzilla query error

Functional Automation

Bug Triage

Aurora Merge

Automated Testing

Manual Testing

Plug-in Compatibility

Web Compatibility

OSX 10.5 EOL

Feature Sign-offs

Update Testing

Bug Triage

Beta 1

Automated Testing

Manual Testing

Update Testing

Feature Sign-off

Build #2 Verifications

Beta 2

Automated Testing

Manual Testing

Update Testing

Feature Sign-off

Beta 3

Automated Testing

Manual Testing

Bug Fix Verifications

Update Testing

Beta 4

Automated Testing

Manual Testing

Bug Fix Verifications

Update Testing

Beta 5

Automated Testing

Manual Testing

Regression Tests

Social API

Click-to-Play

Bug Fix Verifications

Update Testing

Beta 6

Automated Testing

Manual Testing

Regression Tests

Awesomebar

Private Browsing

Bug Fix Verifications

Update Testing

Release Candidate

Automated Testing

Manual Testing

Regression Tests

Bug Fix Verifications

Update Testing

17.0.1

Automated Testing

Manual Testing

Regression Tests

Bug Fix Verifications

Update Testing

Bugs

Triage

New Issues

Known Issues

Navigation menu

Search