MozillaWiki

Releases/Firefox 16/Test Plan: Difference between revisions

Page Discussion

Releases/Firefox 16/Test Plan (view source)

Revision as of 09:57, 1 November 2012

84,250 bytes added ,  1 November 2012
→‎Previously Known
 
(354 intermediate revisions by 10 users not shown)
Line 9: Line 9:
  |-
  |-
  | '''Status'''
  | '''Status'''
  | Currently in '''Nightly''' -- merging to Aurora on July 17, 2012
  | Currently in '''Beta''' -- merging to Release on October 8, 2012
  |-
  |-
  | '''Lead'''
  | '''Lead'''
Line 18: Line 18:
  |-
  |-
  | '''Builds'''
  | '''Builds'''
  | [ftp] [http] [web]
  | [https://www.mozilla.org/beta/ beta.mozilla.org]
  |}
  |}


Line 25: Line 25:
''If you have some free time, please pick a task below and get in touch with one of the leads.''
''If you have some free time, please pick a task below and get in touch with one of the leads.''


== Details ==
== Schedule ==
The following is a description of the tasks which need to be completed throughout this release.
The following is a description of the tasks which need to be completed throughout this release.
* 2012-06-04: {{|}} Nightly Merge
* 2012-06-04: {{done|}} Nightly Merge
* 2012-07-17: {{|}} Aurora Merge
* 2012-07-17: {{done|}} Aurora Merge
* 2012-08-28: {{|}} Pre-Beta feature sign-offs
* 2012-08-28: {{done|}} Pre-Beta feature sign-offs
* 2012-08-30: {{|}} Beta 1
* 2012-08-30: {{done|}} Beta 1
* 2012-09-06: {{|}} Beta 2
* 2012-09-06: {{done|}} Beta 2
* 2012-09-13: {{|}} Beta 3
* 2012-09-13: {{ok|}} Beta 3
* 2012-09-20: {{|}} Beta 4
* 2012-09-20: {{|}} Beta 4
* 2012-09-27: {{|}} Beta 5
* 2012-09-27: {{|}} Beta 5
Line 48: Line 48:
''See the [https://wiki.mozilla.org/Features/Release_Tracking Feature Release Tracking] wiki for a general overview''
''See the [https://wiki.mozilla.org/Features/Release_Tracking Feature Release Tracking] wiki for a general overview''


; {{done|}}
{| class="querytable"
 
| class="header" | Feature
; {{ok|}}
| class="header" | Owner
 
| class="header" | Status
* [[User:P.A./Panel-based Download Manager|Panel-based Download Manager]] (Simona Badau)
| class="header" | Tests
* [[OS X 10.7 support|OS X 10.7 support]] (Vlad Ghetiu)
|-
* [[Opt-in activation for plugins|Opt-in activation for plugins]] (Paul Silaghi)
| [[Platform/Features/Incremental GC|Incremental GC]]  
* [[Speedy Session Restore|Speedy Session Restore]] (Paul Silaghi)
| Ioana Budnar
* [[DevTools/Features/DeveloperToolbar|Developer Toolbar]] (Virgil Dicu)
| On track
* [[DevTools/Features/GCLI|Command Line]] (Mihaela Velimiroviciu)
|
* [[Show PDF inline|Show PDF inline]] (Mihaela Velimiroviciu)
|-
 
| [[Opt-in activation for plugins|Opt-in activation for plugins]]  
; {{risk|}}
| Paul Silaghi
 
| On track
; {{miss|}}
|
|-
| [[DevTools/Features/DeveloperToolbar|Developer Toolbar]]  
| Virgil Dicu
| On track
|
|-
| [[DevTools/Features/GCLI|Command Line]]
| Mihaela Velimiroviciu
| On track
|
|-
| [[Show PDF inline|Show PDF inline]]
| Mihaela Velimiroviciu
| Disabled until Fx18({{bug|786012}})
|
|-
| [[User:P.A./Panel-based Download Manager|Panel-based Download Manager]]
| Simona Badau
| Delayed to Fx18
|
|-
| [[OS X 10.7 support|OS X 10.7 support]]
| Vlad Ghetiu
| Not implemented
|
|-
|}


=== Endurance Automation ===
=== Endurance Automation ===
Line 112: Line 139:
{| class="querytable" style="width:50%"
{| class="querytable" style="width:50%"
  | class="header" | Platform
  | class="header" | Platform
  | class="header" | Result: [http://mozmill-release.blargon7.com/#/functional/reports?branch=12.0&platform=All&from=2012-02-02&to=2012-02-03 report]
  | class="header" | Result: [http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-07-18&to=2012-07-20 report]
  |-
  |-
  | Windows 2000
  | Windows 2000
  | '''Result'''
  | '''PASS''' - 2012-07-19
  |-
  |-
  | Windows XP
  | Windows XP
  | '''Result'''
  | '''PASS''' - 2012-07-19
  |-
  |-
  | Windows Vista
  | Windows Vista
  | '''Result'''
  | '''PASS''' - 2012-07-19
  |-
  |-
  | Windows 7 32-bit
  | Windows 7 32-bit
  | '''Result'''
  | '''PASS''' - 2012-07-19
  |-
  |-
  | Windows 7 64-bit
  | Windows 7 64-bit
  | '''Result'''
  | '''PASS''' - 2012-07-19
  |-
  |-
  | Mac OS X 10.6
  | Mac OS X 10.6
  | '''Result'''
  | '''PASS''' - 2012-07-19
  |-
  |-
  | Mac OS X 10.7
  | Mac OS X 10.7
  | '''Result'''
  | '''PASS''' - 2012-07-19
  |-
  |-
  | Ubuntu 11.04 32-bit
  | Ubuntu 11.04 32-bit
  | '''Result'''
  | '''PASS''' - 2012-07-19
  |-
  |-
  | Ubuntu 11.04 64-bit
  | Ubuntu 11.04 64-bit
  | '''Result'''
  | '''PASS''' - 2012-07-19
  |}
  |}


; Issues Found
;Known Issues
<bugzilla>
{
"id": "719982, 770117"
}
</bugzilla>


=== Updates ===
=== Updates ===
Line 181: Line 213:
  | class="header" | Linux
  | class="header" | Linux
  |-
  |-
  |[https://wiki.mozilla.org/User:P.A./Panel-based_Download_Manager Show PDF Inline Panel Based Download Manager]
  |[https://wiki.mozilla.org/User:P.A./Panel-based_Download_Manager Panel Based Download Manager]
  | Simona Badau
  | Simona Badau
  | '''Results''' - 2012-07-19
  | '''PASS''' - 2012-07-20
  | '''Results''' - 2012-07-19
  | '''PASS''' - 2012-07-19
  | '''Results''' - 2012-07-19
  | '''PASS''' - 2012-07-20
  |-
  |-
  |[https://wiki.mozilla.org/OS_X_10.7_support OS X 10.7 support]
  |[https://wiki.mozilla.org/OS_X_10.7_support OS X 10.7 support] - Not landed
  | Vlad Ghetiu
  | Vlad Ghetiu
  | '''Results''' - 2012-07-19
  | '''N/A''' - 2012-07-19
  | '''Results''' - 2012-07-19
  | '''N/A''' - 2012-07-19
  | '''Results''' - 2012-07-19
  | '''N/A''' - 2012-07-19
  |-
  |-
  |[https://wiki.mozilla.org/Opt-in_activation_for_plugins Opt-in activation for plugins]
  |[https://wiki.mozilla.org/Opt-in_activation_for_plugins Opt-in activation for plugins]
  | Paul Silaghi
  | Paul Silaghi
  | '''Results''' - 2012-07-19
  | '''PASS''' - 2012-07-20
  | '''Results''' - 2012-07-19
  | '''PASS''' - 2012-07-20
  | '''Results''' - 2012-07-19
  | '''PASS''' - 2012-07-20
  |-
  |-
  |[https://wiki.mozilla.org/Speedy_Session_Restore Speedy Session Restore]
  |[https://wiki.mozilla.org/Speedy_Session_Restore Speedy Session Restore]
  | Paul Silaghi
  | Paul Silaghi
  | '''Results''' - 2012-07-19
  | '''PASS''' - 2012-07-19
  | '''Results''' - 2012-07-19
  | '''PASS''' - 2012-07-19
  | '''Results''' - 2012-07-19
  | '''PASS''' - 2012-07-19
  |-
  |-
  |[https://wiki.mozilla.org/DevTools/Features/DeveloperToolbar Developer Toolbar]
  |[https://wiki.mozilla.org/DevTools/Features/DeveloperToolbar Developer Toolbar]
  | Virgil Dicu
  | Virgil Dicu
  | '''Results''' - 2012-07-19
  | '''PASS''' - 2012-07-19
  | '''Results''' - 2012-07-19
  | '''PASS''' - 2012-07-19
  | '''Results''' - 2012-07-19
  | '''PASS''' - 2012-07-19
  |-
  |-
  |[https://wiki.mozilla.org/DevTools/Features/GCLI Command Line]
  |[https://wiki.mozilla.org/DevTools/Features/GCLI Command Line]
| Mihaela Velimiroviciu
| '''PASS''' - 2012-07-20
| '''PASS''' - 2012-07-20
| '''PASS''' - 2012-07-20
|-
|[https://wiki.mozilla.org/Show_PDF_inline Show PDF Inline]
| Mihaela Velimiroviciu
| '''PASS''' - 2012-07-20
| '''PASS''' - 2012-07-20
| '''PASS''' - 2012-07-20
|-
|[[Platform/Features/Incremental GC/TestPlan|
Incremental Garbage Collection]]
| Ioana Budnar
| '''PASS''' - 2012-07-19
| '''PASS''' - 2012-07-20
| '''PASS''' - 2012-07-20
|-
|}
'''Notes:'''
*OS X 10.7 support - not landed
====Smoke tests====
Run the smoketests in MozTrap and report results below.
{|class="querytable" style="width:60%"
|-
| class="header" | Platform
| class="header" | Assignee
| class="header" | Result: [https://moztrap.mozilla.org/results/cases/?filter-run=62 report]
|-
| Windows XP
| Simona
| '''PASS''' - 2012-07-19
|-
| Windows 7 64-bit
| Paul
| '''PASS''' - 2012-07-20
|-
| Mac OS X 10.7.4
| VladG
| '''PASS''' - 2012-07-19
|-
| Ubuntu 12.04 32-bit
| Virgil
| '''PASS''' - 2012-07-19
|-
|}
====Known Issues====
<bugzilla>
{
"id": "692446, 775491"
}
</bugzilla>
===Pre-beta Feature testing and Sign-off===
Manual spot Checks [spreadsheet]
*Platforms: Windows XP, Windows 7 x64, Ubuntu 11.10, Mac 10.6
*Found known issues:
*New issues:
*Status:
Run the '''Firefox 16 Aurora Merge Feature Sign-Off''' tests in [https://moztrap.mozilla.org/runtests/ MozTrap] and report results below.
{| class="querytable"
| class="header" | Feature
| class="header" | Owner
| class="header" | Win32
| class="header" | Mac
| class="header" | Linux
|-
|[https://wiki.mozilla.org/In-content_preferences In-content_preferences]
| Mihaela Velimiroviciu
| '''disabled''' - 2012-08-24
| '''disabled''' - 2012-08-24
| '''disabled''' - 2012-08-24
|-
|[https://wiki.mozilla.org/User:P.A./Panel-based_Download_Manager Panel Based Download Manager]
| Simona Badau
| '''disabled''' - 2012-08-24
| '''disabled''' - 2012-08-24
| '''disabled''' - 2012-08-24
|-
|[https://wiki.mozilla.org/OS_X_10.7_support OS X 10.7 support]
| Vlad Ghetiu
| '''NA''' - 2012-08-24
| '''NA''' - 2012-08-24
| '''NA''' - 2012-08-24
|-
|[https://wiki.mozilla.org/Opt-in_activation_for_plugins Opt-in activation for plugins]
| Paul Silaghi
| '''disabled''' - 2012-08-24
| '''disabled''' - 2012-08-24
| '''disabled''' - 2012-08-24
|-
|[https://wiki.mozilla.org/Speedy_Session_Restore Speedy Session Restore]
  | Paul Silaghi
  | Paul Silaghi
  | '''Results''' - 2012-07-19
  | '''NA''' - 2012-08-24
  | '''Results''' - 2012-07-19
  | '''NA''' - 2012-08-24
  | '''Results''' - 2012-07-19
  | '''NA''' - 2012-08-24
|-
|[https://wiki.mozilla.org/DevTools/Features/DeveloperToolbar Developer Toolbar]
| Virgil Dicu
| '''PASS''' - 2012-08-24
| '''PASS''' - 2012-08-24
| '''PASS''' - 2012-08-24
|-
|[https://wiki.mozilla.org/DevTools/Features/GCLI Command Line]
| Mihaela Velimiroviciu
| '''PASS''' - 2012-08-24
| '''PASS''' - 2012-08-24
| '''PASS''' - 2012-08-24
  |-
  |-
  |[https://wiki.mozilla.org/Show_PDF_inline Show PDF Inline]
  |[https://wiki.mozilla.org/Show_PDF_inline Show PDF Inline]
  | Mihaela Velimiroviciu
  | Mihaela Velimiroviciu
  | '''Results''' - 2012-07-19
  | '''PASS''' - 2012-08-24
  | '''Results''' - 2012-07-19
  | '''PASS''' - 2012-08-24
  | '''Results''' - 2012-07-19
  | '''PASS''' - 2012-08-24
  |-
  |-
  |[[Platform/Features/Incremental GC/TestPlan|
  |[[Platform/Features/Incremental GC/TestPlan|
Incremental Garbage Collection]]
Incremental Garbage Collection]]
  | Ioana Budnar
  | Ioana Budnar
  | '''Results''' - 2012-07-19
  | '''PASS''' - 2012-08-24
  | '''Results''' - 2012-07-19
  | '''PASS''' - 2012-08-24
  | '''Results''' - 2012-07-19
  | '''PASS''' - 2012-08-24
  |-
  |-
  |}
  |}


'''Notes:'''
'''Notes:'''
*
* OS X 10.7 support: Not yet implemented.
* Speedy Session Restore: backed out.


====Smoke tests====
====Smoke tests====
Line 242: Line 388:
  | class="header" | Platform
  | class="header" | Platform
  | class="header" | Assignee
  | class="header" | Assignee
  | class="header" | Result: [https://moztrap.mozilla.org/results/cases/?filter-run=24 report]
  | class="header" | Result: [https://moztrap.mozilla.org/results/cases/?filter-run=62 report]
|-
| Windows XP
| Simona
| '''PASS''' - 2012-08-24
|-
| Windows 7 64-bit
| VladG
| '''PASS''' - 2012-08-24
|-
| Mac OS X 10.7.4
| VladG
| '''PASS''' - 2012-08-24
|-
| Ubuntu 12.04 32-bit
| Virgil
| '''PASS''' - 2012-08-24
|-
|}
 
====Known Issues====
<bugzilla>
{
"id": "775082, 784289"
}
</bugzilla>
 
== Beta 1 ==
{| class="querytable" style="width:30%"
| style="width:10%" |
| style="width:20%" |
|-
| '''Builds'''
| [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b1-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b1-candidates/build1/ HTTP]
|-
| '''ID'''
| 20120828083259
|-
| '''Changeset'''
| [http://hg.mozilla.org/releases/mozilla-beta/rev/c4e1ca772267 c4e1ca772267] ([http://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=2099c2bea208&tochange=c4e1ca772267 pushlog])
|}
 
; Checklist
* {{done|}} [[Releases/Firefox_16/Test_Plan#Automated_Testing|Automated Testing]]
* {{done|}} [[Releases/Firefox_16/Test_Plan#Manual_Testing|Manual Testing]]
* {{done|}} [[Releases/Firefox_16/Test_Plan#Update_Testing|Update Testing]]
* {{done|}} [[Releases/Firefox_16/Test_Plan#Bug_Triage|Bug Triage]]
* {{done|}} Peer Sign-off
* {{ |}} Final sign-off sent and updates qualified on Beta channel
 
=== Automated Testing ===
The following are the detailed results of the automated Mozmill test runs.
{|class="querytable" style="width:50%"
|-
| class="header" style="width:20%" | Platform
| class="header" style="width:30%" | PASS: [http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-08-28&to=2012-08-28 report]
|-
| Windows XP
| '''PASS''' - 2012-08-28
|-
| Windows Vista
| '''PASS''' - 2012-08-28
|-
| Windows 7 32-bit
| '''PASS''' - 2012-08-28
|-
| Windows 7 64-bit
| '''PASS''' - 2012-08-28
|-
| Windows 8 32-bit
| '''N/A''' (pending IT)
|-
| Windows 8 64-bit
| '''N/A''' (pending IT)
|-
| Mac OS X 10.6
| '''PASS''' - 2012-08-28
|-
| Mac OS X 10.7
| '''PASS''' - 2012-08-28
|-
| Mac OS X 10.8
| '''PASS''' - 2012-08-28
|-
| Ubuntu 12.04 32-bit
| '''PASS''' - 2012-08-28
|-
| Ubuntu 12.04 64-bit
| '''PASS''' - 2012-08-28
|}
 
; Test Failures ([https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA known bugs])
 
=== Manual Testing ===
{| class="querytable"
| class="header" | Operating System
| class="header" | Tester
| class="header" | Plug-in Compatibility
| class="header" | Add-on Compatibility
| class="header" | Web Compatibility
|-
| Windows XP
| Simona
| '''PASS''' - 2012-08-29
| '''PASS''' - 2012-08-29
| '''PASS''' - 2012-08-29
|-
| Windows 7
| Mihaela
| '''PASS''' - 2012-08-29
| '''PASS''' - 2012-08-29
| '''PASS''' - 2012-08-29
|-
| Windows 8
| Paul
|'''PASS''' - 2012-08-29
|'''PASS''' - 2012-08-29
|'''PASS''' - 2012-08-29
|-
| Mac OS X
| VladG
| '''PASS''' - 2012-08-29
| '''PASS''' - 2012-08-29
| '''PASS''' - 2012-08-29
|-
| Ubuntu 12.04
| Virgil
| '''PASS''' - 2012-08-29
| '''PASS''' - 2012-08-29
| '''PASS''' - 2012-08-29
|-
| class="header" | Operating System
| class="header" | Tester
| class="header" | Web Mail
| class="header" | Mac 10.8 Signing
| class="header" | Breakpad
|-
| Windows XP
| Simona
| '''PASS''' - 2012-08-29
| '''N/A'''
| '''PASS''' - 2012-08-29
|-
| Windows 7
| Mihaela
| '''PASS''' - 2012-08-29
| '''N/A'''
| '''PASS''' - 2012-08-29
|-
| Windows 8
| Paul
| '''PASS''' - 2012-08-29
| '''N/A'''
| '''PASS''' - 2012-08-29
|-
| Mac OS X
| VladG
| '''PASS''' - 2012-08-29
| '''PASS''' - 2012-08-29
| '''PASS''' - 2012-08-29
|-
| Ubuntu 12.04
| Virgil
| '''PASS''' - 2012-08-29
| '''N/A'''
| '''PASS''' - 2012-08-29
|}
 
==== Plug-in Compatibility ====
Install the following plug-ins and go to websites which use them, looking for rendering issues, functional issues, performance issues, and crashes.
* Adobe Flash Player
**[http://get.adobe.com/flashplayer/ 11.3 Release]
**[http://labs.adobe.com/downloads/flashplayer11-4.html Flash 11.4 beta] (make sure you can autoupdate)
* [http://www.microsoft.com/silverlight/ Microsoft Silverlight]
* [http://www.real.com/ RealPlayer]
** Real Player plugin crashed when opening pages with content managed by the plugin: [http://www.slp3d2.com/launchsiteglobal/test.cfm click], [http://cf.unc.edu/skillstest/take/realplayer.htm click] - {{bug|667399}}
* [http://www.apple.com/quicktime/download/ QuickTime]
 
==== Add-on Compatibility ====
Test the following websites for rendering issues, functional issues, performance issues, and crashes. Pay particular attention to any problems seen after updating Firefox with add-ons installed.
* [https://addons.mozilla.org/en-US/firefox/compatibility/15.0 Top AMO add-ons]
* AVG toolbar (installed via AVG software)
** AVG Safe Search 12.0.0.1912 is not compatible with Firefox 15 beta.
** AVG Security Toolbar can't be installed on Win XP - we get a blue screen while installing it.
* Norton toolbar (installed via Norton software)
** Not currently available. Investigating if we can get this software.
* [http://sg.toolbar.yahoo.com/ Yahoo toolbar]
* [http://bar.yandex.ru/en/ Yandex toolbar]
* [http://babylon-toolbar.software.informer.com/download/ Babylon toolbar]
* [http://lavasoft.com/ Ad-Aware Antivirus]
** The toolbar can't be installed on Firefox.
 
==== Web Compatibility ====
Test the following websites for rendering issues, functional issues, performance issues, and crashes
# [http://www.alexa.com/topsites Overall Top 10]
# [http://www.alexa.com/topsites/countries Top by Country] (pick a random site from three random countries)
# [http://www.alexa.com/topsites/category Top by Category] (pick a random site from three random categories)
# Webmail clients like gmail, hotmail, and yahoo mail (pay particular attention to any built in web messenger software)
# Try Google's [http://support.google.com/maps/bin/answer.py?hl=en&answer=1630790 MapsGL demo]
# Try YouTube's [http://www.youtube.com/html5 HTML5 demo]
 
==== Web Mail ====
Test the following websites for rendering issues, functional issues, performance issues, and crashes. In particular make sure email, attachments, web messenger, and video chat are working as expected.
# [https://mail.google.com/ Google Mail]
# [https://mail.yahoo.com/ Yahoo Mail]
# [https://www.hotmail.com Hotmail]
# [https://www.outlook.com/ Outlook.com]
 
==== Mac 10.8 Signing ====
NOTE: Test needs to be run from a clean Mac OSX 10.8 install. Please keep a vanilla image handy so you can restore once completing this test ([https://people.mozilla.com/~stmichaud/Gatekeeper/MLGatekeeper.html more info]).
* Download the .dmg file and drag the Firefox.app file to a location on your computer
* Open a terminal window and cd to the folder Firefox.app is located
* Run ''codesign -vv Firefox.app'' in terminal, output should contain ''valid on disk'' and ''satisfies its Designated Requirement''
* Run ''xattr Firefox.app'' in terminal, output should contain ''com.apple.quarantine''
* Double click the Firefox.app icon, you should see a warning about the program being provided by an ''unknown developer'' or having been ''downloaded from the Internet''
* Click "open" and browse to any website, Firefox should start and you should be able to browse
* Quit Firefox and double click the Firefox.app icon again, you should ''not'' be reprompted for the warning in step 5, firefox should start up and browse without any issues
 
Spot check build after this test
* Try all the menu item functions
* Gestures
* Plug-in compatibility
* Top-site compatibility
* Add-on compatibility
* Software update
 
==== Breakpad End-to-End ====
Test the end-to-end functionality in the crash reporting feature, across operating systems.
* Use the [http://ted.mielczarek.org/mozilla/crashme.html Crash Me add-on] to trigger a crash
* Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc
* Repeatedly crash Firefox and submit your reports
* Verify that your session restores properly
* Verify your crash reports are submitted properly by going through about:crashes and clicking the reports
* Try adding comments to your crash report and verify the crash-stats.com report shows the comment
 
=== Update Testing ===
* Partial: 15.0b6
* Complete: 15.0b5 and earlier
 
{|class="querytable" style="width:60%"
|-
| class="header" | Platform
| class="header" | Releasetest: [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-08-28&to=2012-08-30&target=16.0 report]
| class="header" | Beta: [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-08-28&to=2012-08-30&target=16.0 report]
|-
| Windows XP
| '''PASS''' - 2012-08-29
| '''PASS''' - 2012-08-30
|-
| Windows Vista
| '''PASS''' - 2012-08-29
| '''PASS''' - 2012-08-30
|-
| Windows 7 32-bit
| '''PASS''' - 2012-08-29
| '''PASS''' - 2012-08-30
|-
| Windows 7 64-bit
| '''PASS''' - 2012-08-29
| '''PASS''' - 2012-08-30
|-
| Windows 8 32-bit
| '''manual''' - see below
| '''manual''' - see below
|-
| Windows 8 64-bit
| '''manual''' - see below
| '''manual''' - see below
|-
| Mac OS X 10.6
| '''PASS''' - 2012-08-29
| '''PASS''' - 2012-08-30
|-
| Mac OS X 10.7
| '''PASS''' - 2012-08-29
| '''PASS''' - 2012-08-30
|-
| Mac OS X 10.8
| '''PASS''' - 2012-08-29
| '''PASS''' - 2012-08-30
|-
| Ubuntu 12.04 32-bit
| '''PASS''' - 2012-08-29
| '''PASS''' - 2012-08-30
|-
| Ubuntu 12.04 64-bit
| '''PASS''' - 2012-08-29
| '''PASS''' - 2012-08-30
|}
 
''Windows 8: manual spotcheck until automation in place''
* RELEASETEST
** '''PASS''' - Firefox 15.0b6 en-US (partial)
** '''PASS''' - Firefox 15.0b6 de (partial)
** '''PASS''' - Firefox 14.0b6 en-US (complete)
** '''PASS''' - Firefox 14.0b6 zh-TW (complete)
* BETA
** '''PASS''' - Firefox 15.0b6 en-US (partial)
** '''PASS''' - Firefox 15.0b6 de (partial)
** '''PASS''' - Firefox 14.0b6 en-US (complete)
** '''PASS''' - Firefox 14.0b6 zh-TW (complete)
 
; Issues
 
=== Bug Triage ===
# [http://bit.ly/RquQy9 VERIFYME] (3): verify the fix and do some exploratory testing around the issue
# [http://bit.ly/Ppe9E0 FIXED] (221): triage these bugs and test to ensure the fix is effective
# [http://bit.ly/PMaYVx QAWANTED] (3): triage these bugs and test to fulfill the QA request
# [http://bit.ly/OtEqj1 UNCONFIRMED] (3): triage these bugs for potential new regressions
 
; Results
* [http://bit.ly/MXYdqS VERIFIED]: 50/221
* [http://bit.ly/POyxuA WONTVERIFY]: 21/221
* [http://bit.ly/O09WSK HELPWANTED]:  14/221
 
== Beta 2 ==
{| class="querytable" style="width:30%"
| style="width:10%" |
| style="width:20%" |
|-
| '''Builds'''
| [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b2-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b2-candidates/build1/ HTTP]
|-
| '''ID'''
| 20120904124322
|-
| '''Changeset'''
| [http://hg.mozilla.org/releases/mozilla-beta/rev/cad589192f3c cad589192f3c] ([http://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=c4e1ca772267&tochange=cad589192f3c pushlog])
|}
 
; Checklist
* {{done|}} [[Releases/Firefox_16/Test_Plan#Automated_Testing_2|Automated Testing]]
* {{done|}} [[Releases/Firefox_16/Test_Plan#Manual_Testing_2|Manual Testing]]
* {{done|}} [[Releases/Firefox_16/Test_Plan#Update_Testing_2|Update Testing]]
* {{done|}} [[Releases/Firefox_16/Test_Plan#Bug_Triage_2|Bug Triage]]
* {{done|}} Peer Sign-off
* {{ |}} Final sign-off sent and updates qualified on Beta channel
 
=== Automated Testing 2 ===
The following are the detailed results of the automated Mozmill test runs.
{|class="querytable" style="width:50%"
|-
| class="header" style="width:20%" | Platform
| class="header" style="width:30%" | PASS: [http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-09-04&to=2012-09-05 report]
|-
| Windows XP
| '''PASS''' - 2012-09-05
|-
| Windows Vista
| '''PASS''' - 2012-09-05
|-
| Windows 7 32-bit
| '''PASS''' - 2012-09-05
|-
| Windows 7 64-bit
| '''PASS''' - 2012-09-05
|-
| Windows 8 32-bit
| '''N/A''' (pending IT)
|-
| Windows 8 64-bit
| '''N/A''' (pending IT)
|-
| Mac OS X 10.6
| '''PASS''' - 2012-09-05
|-
| Mac OS X 10.7
| '''PASS''' - 2012-09-05
|-
| Mac OS X 10.8
| '''PASS''' - 2012-09-05
|-
| Ubuntu 12.04 32-bit
| '''PASS''' - 2012-09-05
|-
| Ubuntu 12.04 64-bit
| '''PASS''' - 2012-09-05
|}
 
; Test Failures ([https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA known bugs])
 
=== Manual Testing 2 ===
{| class="querytable"
| class="header" | Operating System
| class="header" | Tester
| class="header" | Plug-in Compatibility
| class="header" | Add-on Compatibility
| class="header" | Web Compatibility
|-
| Windows XP
| Ioana
| '''PASS''' - 2012-09-07
| '''PASS''' - 2012-09-07
| '''PASS''' - 2012-09-07
|-
| Windows 7
| Mihaela
| '''PASS''' - 2012-09-07
| '''PASS''' - 2012-09-07
| '''PASS''' - 2012-09-07
|-
| Windows 8
| Virgil
| '''PASS''' - 2012-09-07
| '''PASS''' - 2012-09-07
| '''PASS''' - 2012-09-07
|-
| Mac OS X
| VladG
| '''PASS''' - 2012-09-07
| '''PASS''' - 2012-09-07
| '''PASS''' - 2012-09-07
|-
| Ubuntu 12.04
| Ioana
| '''PASS''' - 2012-09-07
| '''PASS''' - 2012-09-07
| '''PASS''' - 2012-09-07
|-
| class="header" | Operating System
| class="header" | Tester
| class="header" | Web Mail
| class="header" | Mac 10.8 Signing
| class="header" | Breakpad
|-
| Windows XP
| Virgil
| '''PASS''' - 2012-09-07
| '''N/A'''
| '''PASS''' - 2012-09-07
|-
| Windows 7
| Mihaela
| '''PASS''' - 2012-09-07
| '''N/A'''
| '''PASS''' - 2012-09-07
|-
| Windows 8
| Virgil
| '''PASS''' - 2012-09-07
| '''N/A'''
| '''PASS''' - 2012-09-07
|-
| Mac OS X
| VladG
| '''PASS''' - 2012-09-07
| '''PASS''' - 2012-09-07
| '''PASS''' - 2012-09-07
|-
| Ubuntu 12.04
| Ioana
| '''PASS''' - 2012-09-07
| '''N/A'''
| '''PASS''' - 2012-09-07
|}
 
==== Plug-in Compatibility 2 ====
Install the following plug-ins and go to websites which use them, looking for rendering issues, functional issues, performance issues, and crashes.
* Adobe Flash Player
**[http://get.adobe.com/flashplayer/ 11.4 Release]
* [http://www.microsoft.com/silverlight/ Microsoft Silverlight]
* [http://www.real.com/ RealPlayer]
** Real Player plugin crashed when opening pages with content managed by the plugin: [http://www.slp3d2.com/launchsiteglobal/test.cfm click], [http://cf.unc.edu/skillstest/take/realplayer.htm click] - {{bug|667399}}
* [http://www.apple.com/quicktime/download/ QuickTime]
 
==== Add-on Compatibility 2 ====
Test the following websites for rendering issues, functional issues, performance issues, and crashes. Pay particular attention to any problems seen after updating Firefox with add-ons installed.
* [https://addons.mozilla.org/en-US/firefox/compatibility/15.0 Top AMO add-ons]
* AVG toolbar (installed via AVG software)
* Norton toolbar (installed via Norton software)
* [http://sg.toolbar.yahoo.com/ Yahoo toolbar]
* [http://bar.yandex.ru/en/ Yandex toolbar]
* [http://babylon-toolbar.software.informer.com/download/ Babylon toolbar]
* [http://lavasoft.com/ Ad-Aware Antivirus]
 
==== Web Compatibility 2 ====
Test the following websites for rendering issues, functional issues, performance issues, and crashes
# [http://www.alexa.com/topsites Overall Top 10]
# [http://www.alexa.com/topsites/countries Top by Country] (pick a random site from three random countries)
# [http://www.alexa.com/topsites/category Top by Category] (pick a random site from three random categories)
# Webmail clients like gmail, hotmail, and yahoo mail (pay particular attention to any built in web messenger software)
# Try Google's [http://support.google.com/maps/bin/answer.py?hl=en&answer=1630790 MapsGL demo]
# Try YouTube's [http://www.youtube.com/html5 HTML5 demo]
 
==== Web Mail 2 ====
Test the following websites for rendering issues, functional issues, performance issues, and crashes. In particular make sure email, attachments, web messenger, and video chat are working as expected.
# [https://mail.google.com/ Google Mail]
# [https://mail.yahoo.com/ Yahoo Mail]
# [https://www.hotmail.com Hotmail]
# [https://www.outlook.com/ Outlook.com]
 
==== Mac 10.8 Signing 2 ====
NOTE: Test needs to be run from a clean Mac OSX 10.8 install. Please keep a vanilla image handy so you can restore once completing this test ([https://people.mozilla.com/~stmichaud/Gatekeeper/MLGatekeeper.html more info]).
* Download the .dmg file and drag the Firefox.app file to a location on your computer
* Open a terminal window and cd to the folder Firefox.app is located
* Run ''codesign -vv Firefox.app'' in terminal, output should contain ''valid on disk'' and ''satisfies its Designated Requirement''
* Run ''xattr Firefox.app'' in terminal, output should contain ''com.apple.quarantine''
* Double click the Firefox.app icon, you should see a warning about the program being provided by an ''unknown developer'' or having been ''downloaded from the Internet''
* Click "open" and browse to any website, Firefox should start and you should be able to browse
* Quit Firefox and double click the Firefox.app icon again, you should ''not'' be reprompted for the warning in step 5, firefox should start up and browse without any issues
 
Spot check build after this test
* Try all the menu item functions
* Gestures
* Plug-in compatibility
* Top-site compatibility
* Add-on compatibility
* Software update
 
==== Breakpad End-to-End 2 ====
Test the end-to-end functionality in the crash reporting feature, across operating systems.
* Use the [http://ted.mielczarek.org/mozilla/crashme.html Crash Me add-on] to trigger a crash
* Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc
* Repeatedly crash Firefox and submit your reports
* Verify that your session restores properly
* Verify your crash reports are submitted properly by going through about:crashes and clicking the reports
* Try adding comments to your crash report and verify the crash-stats.com report shows the comment
 
=== Update Testing 2 ===
* Partial: 16.0b1
* Complete: 15.0b6 and earlier
 
{|class="querytable" style="width:60%"
|-
| class="header" | Platform
| class="header" | Releasetest: [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-09-05&to=2012-09-06&target=16.0 report]
| class="header" | Beta: [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-09-06&to=2012-09-07&target=16.0 report]
|-
| Windows XP
| '''PASS''' - 2012-09-06
| '''PASS''' - 2012-09-06
|-
| Windows Vista
| '''PASS''' - 2012-09-06
| '''PASS''' - 2012-09-06
|-
| Windows 7 32-bit
| '''PASS''' - 2012-09-06
| '''PASS''' - 2012-09-06
|-
| Windows 7 64-bit
| '''PASS''' - 2012-09-06
| '''PASS''' - 2012-09-06
|-
| Windows 8 32-bit
| '''manual''' - see below
| '''manual''' - see below
|-
| Windows 8 64-bit
| '''PASS''' - see below
| '''PASS''' - see below
|-
| Mac OS X 10.6
| '''PASS''' - 2012-09-06
| '''PASS''' - 2012-09-06
|-
| Mac OS X 10.7
| '''PASS''' - 2012-09-06
| '''PASS''' - 2012-09-06
|-
| Mac OS X 10.8
| '''PASS''' - 2012-09-06
| '''PASS''' - 2012-09-06
|-
| Ubuntu 12.04 32-bit
| '''PASS''' - 2012-09-06
| '''PASS''' - 2012-09-06
|-
| Ubuntu 12.04 64-bit
| '''PASS''' - 2012-09-06
| '''PASS''' - 2012-09-06
|}
 
''Windows 8: manual spotcheck until automation in place''
* RELEASETEST
** '''PASS''' - Firefox 16.0b1 en-US (partial)
** '''PASS''' - Firefox 15.0b6 de (complete)
* BETA
** '''PASS''' - Firefox 16.0b1 en-US (partial)
** '''PASS''' - Firefox 15.0b6 de (complete)
 
; Issues
 
=== Bug Triage 2 ===
# [http://bit.ly/RquQy9 VERIFYME] : verify the fix and do some exploratory testing around the issue
# [http://bit.ly/Ppe9E0 FIXED] : triage these bugs and test to ensure the fix is effective
# [http://bit.ly/PMaYVx QAWANTED] : triage these bugs and test to fulfill the QA request
# [http://bit.ly/OtEqj1 UNCONFIRMED] : triage these bugs for potential new regressions
 
; Results
* [http://bit.ly/MXYdqS VERIFIED]: Verified/Fixed Ratio
* [http://bit.ly/POyxuA WONTVERIFY]: Won't Verify/Fixed Ratio
* [http://bit.ly/O09WSK HELPWANTED]:  Help Wanted/Fixed Ratio
 
== Beta 3 ==
{| class="querytable" style="width:30%"
| style="width:10%" |
| style="width:20%" |
|-
| '''Builds'''
| [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b3-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b3-candidates/build1/ HTTP]
|-
| '''ID'''
| 20120911232325
|-
| '''Changeset'''
| [http://hg.mozilla.org/releases/mozilla-beta/rev/de3f06a549de de3f06a549de] ([http://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=cad589192f3c&tochange=de3f06a549de pushlog])
|}
 
; Checklist
* {{done|}} [[Releases/Firefox_16/Test_Plan#Automated_Testing_2|Automated Testing]]
* {{done|}} [[Releases/Firefox_16/Test_Plan#Manual_Testing_2|Manual Testing]]
* {{done|}} [[Releases/Firefox_16/Test_Plan#Update_Testing_2|Update Testing]]
* {{ok|}} [[Releases/Firefox_16/Test_Plan#Bug_Triage_2|Bug Triage]]
* {{done|}} Peer Sign-off
* {{done|}} Final sign-off sent and updates qualified on Beta channel
 
=== Automated Testing 3 ===
The following are the detailed results of the automated Mozmill test runs.
{|class="querytable" style="width:50%"
|-
| class="header" style="width:20%" | Platform
| class="header" style="width:30%" | PASS: [http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-09-11&to=2012-09-12 report]
|-
| Windows XP
| '''PASS''' - 2012-09-12
|-
| Windows Vista
| '''PASS''' - 2012-09-12
|-
| Windows 7 32-bit
| '''PASS''' - 2012-09-12
|-
| Windows 7 64-bit
| '''PASS''' - 2012-09-12
|-
| Windows 8 32-bit
| '''PASS''' - 2012-09-12
|-
| Windows 8 64-bit
| '''PASS''' - 2012-09-12
|-
| Mac OS X 10.6
| '''PASS''' - 2012-09-12
|-
| Mac OS X 10.7
| '''PASS''' - 2012-09-12
|-
| Mac OS X 10.8
| '''PASS''' - 2012-09-12
|-
| Ubuntu 12.04 32-bit
| '''PASS''' - 2012-09-12
|-
| Ubuntu 12.04 64-bit
| '''PASS''' - 2012-09-12
|}
 
; Test Failures ([https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA known bugs])
 
=== Manual Testing 3 ===
{| class="querytable"
| class="header" | Operating System
| class="header" | Tester
| class="header" | Plug-in Compatibility
| class="header" | Add-on Compatibility
| class="header" | Web Compatibility
|-
| Windows XP
| Mihaela
| '''PASS''' - 2012-09-12
| '''PASS''' - 2012-09-13
| '''PASS''' - 2012-09-13
|-
| Windows 7
| Ioana
| '''PASS''' - 2012-09-12
| '''PASS''' - 2012-09-13
| '''PASS''' - 2012-09-13
|-
| Windows 8
| Paul
| '''PASS''' - 2012-09-12
| '''PASS''' - 2012-09-12
| '''PASS''' - 2012-09-12
|-
| Mac OS X
| VladG
| '''PASS''' - 2012-09-12
| '''PASS''' - 2012-09-12
| '''PASS''' - 2012-09-12
|-
| Ubuntu 12.04
| Virgil
| '''PASS''' - 2012-09-12
| '''PASS''' - 2012-09-12
| '''PASS''' - 2012-09-12
|-
| class="header" | Operating System
| class="header" | Tester
| class="header" | Web Mail
| class="header" | Mac 10.8 Signing
| class="header" | Breakpad
|-
| Windows XP
| Mihaela
| '''PASS''' - 2012-09-13
| '''N/A'''
| '''PASS''' - 2012-09-13
|-
| Windows 7
| Ioana
| '''PASS''' - 2012-09-13
| '''N/A'''
| '''PASS''' - 2012-09-13
|-
| Windows 8
| Paul
| '''PASS''' - 2012-09-13
| '''N/A'''
| '''PASS''' - 2012-09-13
|-
| Mac OS X
| VladG
| '''PASS''' - 2012-09-12
| '''PASS''' - 2012-09-12
| '''PASS''' - 2012-09-12
|-
| Ubuntu 12.04
| Virgil
| '''PASS''' - 2012-09-12
| '''N/A'''
| '''PASS''' - 2012-09-12
|}
 
==== Plug-in Compatibility 3 ====
Install the following plug-ins and go to websites which use them, looking for rendering issues, functional issues, performance issues, and crashes.
* Adobe Flash Player
**[http://get.adobe.com/flashplayer/ 11.4 Release]
* [http://www.microsoft.com/silverlight/ Microsoft Silverlight]
* [http://www.real.com/ RealPlayer]
** Real Player plugin crashed when opening pages with content managed by the plugin: [http://www.slp3d2.com/launchsiteglobal/test.cfm click], [http://cf.unc.edu/skillstest/take/realplayer.htm click] - {{bug|667399}}
* [http://www.apple.com/quicktime/download/ QuickTime]
 
==== Add-on Compatibility 3 ====
Test the following websites for rendering issues, functional issues, performance issues, and crashes. Pay particular attention to any problems seen after updating Firefox with add-ons installed.
* [https://addons.mozilla.org/en-US/firefox/compatibility/15.0 Top AMO add-ons]
* AVG toolbar (installed via AVG software)
* Norton toolbar (installed via Norton software)
* [http://sg.toolbar.yahoo.com/ Yahoo toolbar]
* [http://bar.yandex.ru/en/ Yandex toolbar]
* [http://babylon-toolbar.software.informer.com/download/ Babylon toolbar]
* [http://lavasoft.com/ Ad-Aware Antivirus]
 
==== Web Compatibility 3 ====
Test the following websites for rendering issues, functional issues, performance issues, and crashes
# [http://www.alexa.com/topsites Overall Top 10]
# [http://www.alexa.com/topsites/countries Top by Country] (pick a random site from three random countries)
# [http://www.alexa.com/topsites/category Top by Category] (pick a random site from three random categories)
# Webmail clients like gmail, hotmail, and yahoo mail (pay particular attention to any built in web messenger software)
# Try Google's [http://support.google.com/maps/bin/answer.py?hl=en&answer=1630790 MapsGL demo]
# Try YouTube's [http://www.youtube.com/html5 HTML5 demo]
 
==== Web Mail 3 ====
Test the following websites for rendering issues, functional issues, performance issues, and crashes. In particular make sure email, attachments, web messenger, and video chat are working as expected.
# [https://mail.google.com/ Google Mail]
# [https://mail.yahoo.com/ Yahoo Mail]
# [https://www.hotmail.com Hotmail]
# [https://www.outlook.com/ Outlook.com]
 
==== Mac 10.8 Signing 3 ====
NOTE: Test needs to be run from a clean Mac OSX 10.8 install. Please keep a vanilla image handy so you can restore once completing this test ([https://people.mozilla.com/~stmichaud/Gatekeeper/MLGatekeeper.html more info]).
* Download the .dmg file and drag the Firefox.app file to a location on your computer
* Open a terminal window and cd to the folder Firefox.app is located
* Run ''codesign -vv Firefox.app'' in terminal, output should contain ''valid on disk'' and ''satisfies its Designated Requirement''
* Run ''xattr Firefox.app'' in terminal, output should contain ''com.apple.quarantine''
* Double click the Firefox.app icon, you should see a warning about the program being provided by an ''unknown developer'' or having been ''downloaded from the Internet''
* Click "open" and browse to any website, Firefox should start and you should be able to browse
* Quit Firefox and double click the Firefox.app icon again, you should ''not'' be reprompted for the warning in step 5, firefox should start up and browse without any issues
 
Spot check build after this test
* Try all the menu item functions
* Gestures
* Plug-in compatibility
* Top-site compatibility
* Add-on compatibility
* Software update
 
==== Breakpad End-to-End 3 ====
Test the end-to-end functionality in the crash reporting feature, across operating systems.
* Use the [http://ted.mielczarek.org/mozilla/crashme.html Crash Me add-on] to trigger a crash
* Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc
* Repeatedly crash Firefox and submit your reports
* Verify that your session restores properly
* Verify your crash reports are submitted properly by going through about:crashes and clicking the reports
* Try adding comments to your crash report and verify the crash-stats.com report shows the comment
 
=== Update Testing 3 ===
* Partial: 16.0b2
* Complete: 15.0b6 and earlier
 
{|class="querytable" style="width:60%"
|-
| class="header" | Platform
| class="header" | Releasetest: [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-09-11&to=2012-09-13&target=16.0 report]
| class="header" | Beta: [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-09-11&to=2012-09-13&target=16.0 report]
|-
| Windows XP
| '''PASS''' - 2012-09-13
| '''PASS''' - 2012-09-13
|-
| Windows Vista
| '''PASS''' - 2012-09-13
| '''PASS''' - 2012-09-13
|-
| Windows 7 32-bit
| '''PASS''' - 2012-09-13
| '''PASS''' - 2012-09-13
|-
| Windows 7 64-bit
| '''PASS''' - 2012-09-13
| '''PASS''' - 2012-09-13
|-
| Windows 8 32-bit
| '''manual'''-see below - 2012-09-13
| '''Results''' - yyyy-mm-dd
|-
| Windows 8 64-bit
| '''manual'''-see below - 2012-09-13
| '''Results''' - yyyy-mm-dd
|-
| Mac OS X 10.6
| '''PASS''' - 2012-09-13
| '''PASS''' - 2012-09-13
|-
| Mac OS X 10.7
| '''PASS''' - 2012-09-13
| '''PASS''' - 2012-09-13
|-
| Mac OS X 10.8
| '''PASS''' - 2012-09-13
| '''PASS''' - 2012-09-13
|-
| Ubuntu 12.04 32-bit
| '''PASS''' - 2012-09-13
| '''PASS''' - 2012-09-13
|-
| Ubuntu 12.04 64-bit
| '''PASS''' - 2012-09-13
| '''PASS''' - 2012-09-13
|}
 
''Windows 8: manual spotcheck until automation in place''
* RELEASETEST
** '''PASS''' - Firefox 16.0b1 en-US (partial)
** '''PASS''' - Firefox 15.0b6 de (complete)
* BETA
** '''PASS''' - Firefox 16.0b2 en-US (partial)
** '''PASS''' - Firefox 15.0b6 de (complete)
 
; Issues
 
=== Bug Triage 3 ===
# [http://bit.ly/RquQy9 VERIFYME (3)] : verify the fix and do some exploratory testing around the issue
# [http://bit.ly/Ppe9E0 FIXED (184)] : triage these bugs and test to ensure the fix is effective
# [http://bit.ly/PMaYVx QAWANTED (8)] : triage these bugs and test to fulfill the QA request
# [http://bit.ly/OtEqj1 UNCONFIRMED (10)] : triage these bugs for potential new regressions
 
; Results
* [http://bit.ly/MXYdqS VERIFIED]: 37/249
* [http://bit.ly/POyxuA WONTVERIFY]: 23/249
* [http://bit.ly/O09WSK HELPWANTED]: 5/249
 
== Beta 4 ==
{| class="querytable" style="width:30%"
| style="width:10%" |
| style="width:20%" |
|-
| '''Builds'''
| [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b4-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b4-candidates/build1/ HTTP]
|-
| '''ID'''
| 20120919065210
|-
| '''Changeset'''
| [http://hg.mozilla.org/releases/mozilla-beta/rev/c3be659f6121 c3be659f6121]
|}
 
; Checklist
* {{done|}} [[Releases/Firefox_16/Test_Plan#Automated_Testing_4|Automated Testing]]
* {{done|}} [[Releases/Firefox_16/Test_Plan#Manual_Testing_4|Manual Testing]]
* {{done|}} [[Releases/Firefox_16/Test_Plan#Update_Testing_4|Update Testing]]
* {{ok|}} [[Releases/Firefox_16/Test_Plan#Bug_Triage_4|Bug Triage]]
* {{done|}} Peer Sign-off
* {{ | }} Final sign-off sent and updates qualified on Beta channel
 
=== Automated Testing 4 ===
The following are the detailed results of the automated Mozmill test runs.
{|class="querytable" style="width:50%"
|-
| class="header" style="width:20%" | Platform
| class="header" style="width:30%" | RESULT: [http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-09-18&to=2012-09-19 report]
|-
| Windows XP
| '''PASS''' - 2012-09-19
|-
| Windows Vista
| '''PASS''' - 2012-09-19
|-
| Windows 7 32-bit
| '''PASS''' - 2012-09-19
|-
| Windows 7 64-bit
| '''PASS''' - 2012-09-19
|-
| Windows 8 32-bit
| '''PASS''' - 2012-09-19
|-
| Windows 8 64-bit
| '''PASS''' - 2012-09-19
|-
| Mac OS X 10.6
| '''PASS''' - 2012-09-19
|-
| Mac OS X 10.7
| '''PASS''' - 2012-09-19
|-
| Mac OS X 10.8
| '''PASS''' - 2012-09-19
|-
| Ubuntu 12.04 32-bit
| '''PASS''' - 2012-09-19
|-
| Ubuntu 12.04 64-bit
| '''PASS''' - 2012-09-19
|}
 
; Test Failures ([https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA known bugs])
 
=== Manual Testing 4 ===
{| class="querytable"
| class="header" | Operating System
| class="header" | Tester
| class="header" | Plug-in Compatibility
| class="header" | Add-on Compatibility
| class="header" | Web Compatibility
|-
| Windows XP
| Vlad, Mihaela
| '''PASS''' - 2012-09-21
| '''PASS''' - 2012-09-21
| '''PASS''' - 2012-09-21
|-
| Windows 7
| Simona
| '''PASS''' - 2012-09-20
| '''PASS''' - 2012-09-20
| '''PASS''' - 2012-09-20
|-
| Windows 8
| Paul
| '''PASS''' - 2012-09-20
| '''PASS''' - 2012-09-20
| '''PASS''' - 2012-09-20
|-
| Mac OS X
| VladG
| '''PASS''' - 2012-09-20
| '''PASS''' - 2012-09-20
| '''PASS''' - 2012-09-20
|-
| Ubuntu 12.04
| Mihaela
| '''PASS''' - 2012-09-20
| '''PASS''' - 2012-09-20
| '''PASS''' - 2012-09-20
|-
| class="header" | Operating System
| class="header" | Tester
| class="header" | Web Mail
| class="header" | Mac 10.8 Signing
| class="header" | Breakpad
|-
| Windows XP
| Mihaela
| '''PASS''' - 2012-09-20
| '''N/A'''
| '''PASS''' - 2012-09-20
|-
| Windows 7
| Simona
| '''PASS''' - 2012-09-20
| '''N/A'''
| '''PASS''' - 2012-09-20
|-
| Windows 8
| Paul
| '''PASS''' - 2012-09-20
| '''N/A'''
| '''PASS''' - 2012-09-20
|-
| Mac OS X
| VladG
| '''PASS''' - 2012-09-20
| '''PASS''' - 2012-09-20
| '''PASS''' - 2012-09-20
|-
| Ubuntu 12.04
| Mihaela
| '''PASS''' - 2012-09-20
| '''N/A'''
| '''PASS''' - 2012-09-20
|}
 
==== Plug-in Compatibility 4 ====
Install the following plug-ins and go to websites which use them, looking for rendering issues, functional issues, performance issues, and crashes.
* Adobe Flash Player
**[http://get.adobe.com/flashplayer/ 11.4 Release]
* [http://www.microsoft.com/silverlight/ Microsoft Silverlight]
* [http://www.real.com/ RealPlayer]
** Real Player plugin crashed when opening pages with content managed by the plugin: [http://www.slp3d2.com/launchsiteglobal/test.cfm click], [http://cf.unc.edu/skillstest/take/realplayer.htm click] - {{bug|667399}}
* [http://www.apple.com/quicktime/download/ QuickTime]
 
==== Add-on Compatibility 4 ====
Test the following websites for rendering issues, functional issues, performance issues, and crashes. Pay particular attention to any problems seen after updating Firefox with add-ons installed.
* [https://addons.mozilla.org/en-US/firefox/compatibility/15.0 Top AMO add-ons]
* AVG toolbar (installed via AVG software)
* Norton toolbar (installed via Norton software)
* [http://sg.toolbar.yahoo.com/ Yahoo toolbar]
* [http://bar.yandex.ru/en/ Yandex toolbar]
* [http://babylon-toolbar.software.informer.com/download/ Babylon toolbar]
** Babylon toolbar can't be installed on Firefox 16 - {{bug|721264}}
* [http://lavasoft.com/ Ad-Aware Antivirus]
 
==== Web Compatibility 4 ====
Test the following websites for rendering issues, functional issues, performance issues, and crashes
# [http://www.alexa.com/topsites Overall Top 10]
# [http://www.alexa.com/topsites/countries Top by Country] (pick a random site from three random countries)
# [http://www.alexa.com/topsites/category Top by Category] (pick a random site from three random categories)
# Webmail clients like gmail, hotmail, and yahoo mail (pay particular attention to any built in web messenger software)
# Try Google's [http://support.google.com/maps/bin/answer.py?hl=en&answer=1630790 MapsGL demo]
# Try YouTube's [http://www.youtube.com/html5 HTML5 demo]
 
==== Web Mail 4 ====
Test the following websites for rendering issues, functional issues, performance issues, and crashes. In particular make sure email, attachments, web messenger, and video chat are working as expected.
# [https://mail.google.com/ Google Mail]
# [https://mail.yahoo.com/ Yahoo Mail]
# [https://www.hotmail.com Hotmail]
# [https://www.outlook.com/ Outlook.com]
 
==== Mac 10.8 Signing 4 ====
NOTE: Test needs to be run from a clean Mac OSX 10.8 install. Please keep a vanilla image handy so you can restore once completing this test ([https://people.mozilla.com/~stmichaud/Gatekeeper/MLGatekeeper.html more info]).
* Download the .dmg file and drag the Firefox.app file to a location on your computer
* Open a terminal window and cd to the folder Firefox.app is located
* Run ''codesign -vv Firefox.app'' in terminal, output should contain ''valid on disk'' and ''satisfies its Designated Requirement''
* Run ''xattr Firefox.app'' in terminal, output should contain ''com.apple.quarantine''
* Double click the Firefox.app icon, you should see a warning about the program being provided by an ''unknown developer'' or having been ''downloaded from the Internet''
* Click "open" and browse to any website, Firefox should start and you should be able to browse
* Quit Firefox and double click the Firefox.app icon again, you should ''not'' be reprompted for the warning in step 5, firefox should start up and browse without any issues
 
Spot check build after this test
* Try all the menu item functions
* Gestures
* Plug-in compatibility
* Top-site compatibility
* Add-on compatibility
* Software update
 
==== Breakpad End-to-End 4 ====
Test the end-to-end functionality in the crash reporting feature, across operating systems.
* Use the [http://ted.mielczarek.org/mozilla/crashme.html Crash Me add-on] to trigger a crash
* Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc
* Repeatedly crash Firefox and submit your reports
* Verify that your session restores properly
* Verify your crash reports are submitted properly by going through about:crashes and clicking the reports
* Try adding comments to your crash report and verify the crash-stats.com report shows the comment
 
=== Update Testing 4 ===
* Partial: 16.0b3
* Complete: 16.0b2 and earlier
 
{|class="querytable" style="width:60%"
|-
| class="header" | Platform
| class="header" | Releasetest: [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-09-18&to=2012-09-19&target=16.0 report]
| class="header" | Beta: [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-09-18&to=2012-09-19&target=16.0 report]
|-
| Windows XP
| '''PASS''' - 2012-09-20
| '''RESULT''' - 2012-09-19
|-
| Windows Vista
| '''PASS''' - 2012-09-20
| '''RESULT''' - 2012-09-19
|-
| Windows 7 32-bit
| '''PASS''' - 2012-09-20
| '''RESULT''' - 2012-09-19
|-
| Windows 7 64-bit
| '''PASS''' - 2012-09-20
| '''RESULT''' - 2012-09-19
|-
| Windows 8 32-bit
| '''manual'''-see below - 2012-09-19
| '''Results''' - yyyy-mm-dd
|-
| Windows 8 64-bit
| '''manual'''-see below - 2012-09-19
| '''Results''' - yyyy-mm-dd
|-
| Mac OS X 10.6
| '''PASS''' - 2012-09-20
| '''RESULT''' - 2012-09-19
|-
| Mac OS X 10.7
| '''PASS''' - 2012-09-20
| '''RESULT''' - 2012-09-19
|-
| Mac OS X 10.8
| '''PASS''' - 2012-09-20
| '''RESULT''' - 2012-09-19
|-
| Ubuntu 12.04 32-bit
| '''PASS''' - 2012-09-20
| '''RESULT''' - 2012-09-19
|-
| Ubuntu 12.04 64-bit
| '''PASS''' - 2012-09-20
| '''RESULT''' - 2012-09-19
|}
 
''Windows 8: manual spotcheck until automation in place''
* RELEASETEST
** '''PASS''' - Firefox 16.0b3 en-US (partial)
** '''PASS''' - Firefox 15.0b6 de (complete)
* BETA
** '''RESULT''' - Firefox 16.0b3 en-US (partial)
** '''RESULT''' - Firefox 15.0b6 de (complete)
 
; Issues
 
=== Bug Triage 4 ===
# [http://bit.ly/RquQy9 VERIFYME (?)] : verify the fix and do some exploratory testing around the issue
# [http://bit.ly/Ppe9E0 FIXED (?)] : triage these bugs and test to ensure the fix is effective
# [http://bit.ly/PMaYVx QAWANTED (?)] : triage these bugs and test to fulfill the QA request
# [http://bit.ly/OtEqj1 UNCONFIRMED (?)] : triage these bugs for potential new regressions
 
; Results
* [http://bit.ly/MXYdqS VERIFIED]: ratios
* [http://bit.ly/POyxuA WONTVERIFY]: ratios
* [http://bit.ly/O09WSK HELPWANTED]: ratios
 
Verification [https://docs.google.com/spreadsheet/ccc?key=0AoLS2jzcUzsHdEFLcDlFa1ppTWZXRDBvZ3Btamk2MVE#gid=0 Spreadsheet]
 
 
== Beta 5 ==
{| class="querytable" style="width:30%"
| style="width:10%" |
| style="width:20%" |
|-
| '''Builds'''
| [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b5-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b5-candidates/build1/ HTTP]
|-
| '''ID'''
| 20120925201946
|-
| '''Changeset'''
| [https://hg.mozilla.org/releases/mozilla-beta/rev/7c0af9b7ed61 7c0af9b7ed61] ([http://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=FIREFOX_16_0b4_BUILD1&tochange=FIREFOX_16_0b5_BUILD1 pushlog])
|}
 
; Checklist
* {{done|}} [[Releases/Firefox_16/Test_Plan#Automated_Testing_5|Automated Testing]]
* {{done|}} [[Releases/Firefox_16/Test_Plan#Manual_Testing_5|Manual Testing]]
* {{done|}} [[Releases/Firefox_16/Test_Plan#Update_Testing_5|Update Testing]]
* {{ok|}} [[Releases/Firefox_16/Test_Plan#Bug_Triage_5|Bug Triage]]
* {{done|}} Peer Sign-off
* {{ | }} Final sign-off sent and updates qualified on Beta channel
 
=== Automated Testing 5 ===
The following are the detailed results of the automated Mozmill test runs.
{|class="querytable" style="width:50%"
|-
| class="header" style="width:20%" | Platform
| class="header" style="width:30%" | RESULT: [http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-09-25&to=2012-09-26 report]
  |-
  |-
  | Windows XP
  | Windows XP
  | tester
  | '''PASS''' - 2012-09-26
  | '''Results''' - 2012-07-19
|-
| Windows Vista
| '''PASS''' - 2012-09-26
|-
| Windows 7 32-bit
| '''PASS''' - 2012-09-26
|-
| Windows 7 64-bit
| '''PASS''' - 2012-09-26
|-
| Windows 8 32-bit
| '''PASS''' - 2012-09-26
|-
| Windows 8 64-bit
| '''PASS''' - 2012-09-26
|-
| Mac OS X 10.6
| '''PASS''' - 2012-09-26
|-
| Mac OS X 10.7
| '''PASS''' - 2012-09-26
|-
| Mac OS X 10.8
| '''PASS''' - 2012-09-26
|-
| Ubuntu 12.04 32-bit
| '''PASS''' - 2012-09-26
|-
| Ubuntu 12.04 64-bit
| '''PASS''' - 2012-09-26
|}
 
; Test Failures ([https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA known bugs])
 
=== Manual Testing 5 ===
{| class="querytable"
| class="header" | Operating System
| class="header" | Tester
| class="header" | Plug-in Compatibility
| class="header" | Add-on Compatibility
| class="header" | Web Compatibility
|-
| Windows XP
| Simona
| '''PASS''' - 2012-09-26
| '''PASS''' - 2012-09-26
| '''PASS''' - 2012-09-26
|-
| Windows 7
| Paul
| '''PASS''' - 2012-09-26
| '''PASS''' - 2012-09-26
| '''PASS''' - 2012-09-26
|-
| Windows 8
| Mihaela
| '''PASS''' - 2012-09-26
| '''PASS''' - 2012-09-26
| '''PASS''' - 2012-09-26
|-
| Mac OS X
| VladG
| '''PASS''' - 2012-09-26
| '''PASS''' - 2012-09-26
| '''PASS''' - 2012-09-26
|-
| Ubuntu 12.04
| Ioana
| '''PASS''' - 2012-09-26
| '''PASS''' - 2012-09-26
| '''PASS''' - 2012-09-27
|-
| class="header" | Operating System
| class="header" | Tester
| class="header" | Web Mail
| class="header" | Mac 10.8 Signing
| class="header" | Breakpad
|-
| Windows XP
| Simona
| '''PASS''' - 2012-09-26
| '''N/A'''
| '''PASS''' - 2012-09-26
|-
| Windows 7
| Paul
| '''PASS''' - 2012-09-26
| '''N/A'''
| '''PASS''' - 2012-09-26
|-
| Windows 8
| Mihaela
| '''PASS''' - 2012-09-26
| '''N/A'''
| '''PASS''' - 2012-09-27
|-
| Mac OS X
| VladG
| '''PASS''' - 2012-09-27
| '''PASS''' - 2012-09-26
| '''PASS''' - 2012-09-27
|-
| Ubuntu 12.04
| Ioana
| '''PASS''' - 2012-09-27
| '''N/A'''
| '''PASS''' - 2012-09-27
|}
 
==== Plug-in Compatibility 5 ====
Install the following plug-ins and go to websites which use them, looking for rendering issues, functional issues, performance issues, and crashes.
* Adobe Flash Player
**[http://get.adobe.com/flashplayer/ 11.4 Release]
* [http://www.microsoft.com/silverlight/ Microsoft Silverlight]
* [http://www.real.com/ RealPlayer]
** Real Player plugin crashed when opening pages with content managed by the plugin: [http://www.slp3d2.com/launchsiteglobal/test.cfm click], [http://cf.unc.edu/skillstest/take/realplayer.htm click] - {{bug|667399}}
* [http://www.apple.com/quicktime/download/ QuickTime]
 
==== Add-on Compatibility 5 ====
Test the following websites for rendering issues, functional issues, performance issues, and crashes. Pay particular attention to any problems seen after updating Firefox with add-ons installed.
* [https://addons.mozilla.org/en-US/firefox/compatibility/16.0 Top AMO add-ons]
* AVG toolbar (installed via AVG software)
* Norton toolbar (installed via Norton software)
* [http://sg.toolbar.yahoo.com/ Yahoo toolbar]
* [http://bar.yandex.ru/en/ Yandex toolbar]
* [http://babylon-toolbar.software.informer.com/download/ Babylon toolbar]
** Babylon toolbar can't be installed on Firefox 16 - {{bug|721264}}
* [http://lavasoft.com/ Ad-Aware Antivirus]
 
==== Web Compatibility 5 ====
Test the following websites for rendering issues, functional issues, performance issues, and crashes
# [http://www.alexa.com/topsites Overall Top 10]
# [http://www.alexa.com/topsites/countries Top by Country] (pick a random site from three random countries)
# [http://www.alexa.com/topsites/category Top by Category] (pick a random site from three random categories)
# Webmail clients like gmail, hotmail, and yahoo mail (pay particular attention to any built in web messenger software)
# Try Google's [http://support.google.com/maps/bin/answer.py?hl=en&answer=1630790 MapsGL demo]
# Try YouTube's [http://www.youtube.com/html5 HTML5 demo]
 
==== Web Mail 5 ====
Test the following websites for rendering issues, functional issues, performance issues, and crashes. In particular make sure email, attachments, web messenger, and video chat are working as expected.
# [https://mail.google.com/ Google Mail]
# [https://mail.yahoo.com/ Yahoo Mail]
# [https://www.hotmail.com Hotmail]
# [https://www.outlook.com/ Outlook.com]
 
==== Mac 10.8 Signing 5 ====
NOTE: Test needs to be run from a clean Mac OSX 10.8 install. Please keep a vanilla image handy so you can restore once completing this test ([https://people.mozilla.com/~stmichaud/Gatekeeper/MLGatekeeper.html more info]).
* Download the .dmg file and drag the Firefox.app file to a location on your computer
* Open a terminal window and cd to the folder Firefox.app is located
* Run ''codesign -vv Firefox.app'' in terminal, output should contain ''valid on disk'' and ''satisfies its Designated Requirement''
* Run ''xattr Firefox.app'' in terminal, output should contain ''com.apple.quarantine''
* Double click the Firefox.app icon, you should see a warning about the program being provided by an ''unknown developer'' or having been ''downloaded from the Internet''
* Click "open" and browse to any website, Firefox should start and you should be able to browse
* Quit Firefox and double click the Firefox.app icon again, you should ''not'' be reprompted for the warning in step 5, firefox should start up and browse without any issues
 
Spot check build after this test
* Try all the menu item functions
* Gestures
* Plug-in compatibility
* Top-site compatibility
* Add-on compatibility
* Software update
 
==== Breakpad End-to-End 5 ====
Test the end-to-end functionality in the crash reporting feature, across operating systems.
* Use the [http://ted.mielczarek.org/mozilla/crashme.html Crash Me add-on] to trigger a crash
* Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc
* Repeatedly crash Firefox and submit your reports
* Verify that your session restores properly
* Verify your crash reports are submitted properly by going through about:crashes and clicking the reports
* Try adding comments to your crash report and verify the crash-stats.com report shows the comment
 
==== Windows 8 ====
Test Firefox integration with Windows 8 - [https://etherpad.mozilla.org/Win-8-Classic-Test-Plan etherpad].
 
=== Update Testing 5 ===
* Partial: 16.0b4
* Complete: 16.0b3 and earlier
 
{|class="querytable" style="width:60%"
|-
| class="header" | Platform
| class="header" | Releasetest: [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-09-25&to=2012-09-26&target=16.0 report]
| class="header" | Beta: [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-09-25&to=2012-09-26&target=16.0 report]
|-
| Windows XP
| '''PASS''' - 2012-09-26
| '''RESULT''' - 2012-09-26
|-
| Windows Vista
| '''PASS''' - 2012-09-26
| '''RESULT''' - 2012-09-26
|-
| Windows 7 32-bit
| '''PASS''' - 2012-09-26
| '''RESULT''' - 2012-09-26
|-
| Windows 7 64-bit
| '''PASS''' - 2012-09-26
| '''RESULT''' - 2012-09-26
|-
| Windows 8 32-bit
| '''manual'''-see below - 2012-09-19
  | '''Results''' - yyyy-mm-dd
|-
| Windows 8 64-bit
| '''manual'''-see below - 2012-09-19
| '''Results''' - yyyy-mm-dd
|-
| Mac OS X 10.6
| '''PASS''' - 2012-09-26
| '''RESULT''' - 2012-09-26
|-
| Mac OS X 10.7
| '''PASS''' - 2012-09-26
| '''RESULT''' - 2012-09-26
|-
| Mac OS X 10.8
| '''PASS''' - 2012-09-26
| '''RESULT''' - 2012-09-26
|-
| Ubuntu 12.04 32-bit
| '''PASS''' - 2012-09-26
| '''RESULT''' - 2012-09-26
|-
| Ubuntu 12.04 64-bit
| '''PASS''' - 2012-09-26
| '''RESULT''' - 2012-09-26
|}
 
''Windows 8: manual spotcheck until automation in place''
* RELEASETEST
** '''RESULT''' - Firefox 16.0b4 en-US (partial)
** '''RESULT''' - Firefox 15.0b6 de (complete)
* BETA
** '''RESULT''' - Firefox 16.0b4 en-US (partial)
** '''RESULT''' - Firefox 15.0b6 de (complete)
 
; Issues
 
=== Bug Triage 5 ===
# [http://bit.ly/RquQy9 VERIFYME (19)] : verify the fix and do some exploratory testing around the issue
# [http://bit.ly/Ppe9E0 FIXED (184)] : triage these bugs and test to ensure the fix is effective
# [http://bit.ly/PMaYVx QAWANTED (6)] : triage these bugs and test to fulfill the QA request
# [http://bit.ly/OtEqj1 UNCONFIRMED (14)] : triage these bugs for potential new regressions
 
; Results
* [http://bit.ly/MXYdqS VERIFIED]: 81/296
* [http://bit.ly/POyxuA WONTVERIFY]: 30/296
* [http://bit.ly/O09WSK HELPWANTED]: 5/296
 
== Beta 6 ==
{| class="querytable" style="width:30%"
| style="width:10%" |
| style="width:20%" |
|-
| '''Builds'''
| [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b6-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b6-candidates/build1/ HTTP]
|-
| '''ID'''
| 20121002073616
|-
| '''Changeset'''
| [https://hg.mozilla.org/releases/mozilla-beta/rev/8a8a932fbef4 8a8a932fbef4] ([http://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=FIREFOX_16_0b5_BUILD1&tochange=FIREFOX_16_0b6_BUILD1 pushlog])
|}
 
; Checklist
* {{done|}} [[Releases/Firefox_16/Test_Plan#Automated_Testing_6|Automated Testing]]
* {{done|}} [[Releases/Firefox_16/Test_Plan#Manual_Testing_6|Manual Testing]]
* {{done|}} [[Releases/Firefox_16/Test_Plan#Update_Testing_6|Update Testing]]
* {{ok|}} [[Releases/Firefox_16/Test_Plan#Bug_Triage_6|Bug Triage]]
* {{ok|}} Peer Sign-off
* {{ | }} Final sign-off sent and updates qualified on Beta channel
 
=== Automated Testing 6 ===
The following are the detailed results of the automated Mozmill test runs.
{|class="querytable" style="width:50%"
|-
| class="header" style="width:20%" | Platform
| class="header" style="width:30%" | RESULT: [http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-10-01&to=2012-10-02 report]
|-
| Windows XP
| '''PASS''' - 2012-10-02
|-
| Windows Vista
| '''PASS''' - 2012-10-02
|-
| Windows 7 32-bit
| '''PASS''' - 2012-10-02
|-
| Windows 7 64-bit
| '''PASS''' - 2012-10-02
|-
| Windows 8 32-bit
| '''PASS''' - 2012-10-02
|-
| Windows 8 64-bit
| '''PASS''' - 2012-10-02
|-
| Mac OS X 10.6
| '''PASS''' - 2012-10-02
|-
| Mac OS X 10.7
| '''PASS''' - 2012-10-02
|-
| Mac OS X 10.8
| '''PASS''' - 2012-10-02
|-
| Ubuntu 12.04 32-bit
| '''PASS''' - 2012-10-02
|-
| Ubuntu 12.04 64-bit
| '''PASS''' - 2012-10-02
|}
 
; Test Failures ([https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA known bugs])
 
=== Manual Testing 6 ===
{| class="querytable"
| class="header" | Operating System
| class="header" | Tester
| class="header" | Plug-in Compatibility
| class="header" | Add-on Compatibility
| class="header" | Web Compatibility
|-
| Windows XP
| Paul/Virgil
| '''PASS''' - 2012-10-02
| '''PASS''' - 2012-10-02
| '''PASS''' - 2012-10-02
|-
| Windows 7
| Ioana
| '''PASS''' - 2012-10-02
| '''PASS''' - 2012-10-02
| '''PASS''' - 2012-10-02
|-
| Windows 8
| Simona
| '''PASS''' - 2012-10-02
| '''PASS''' - 2012-10-02
| '''PASS''' - 2012-10-02
|-
| Mac OS X
| VladG
| '''PASS''' - 2012-10-03
| '''PASS''' - 2012-10-03
| '''PASS''' - 2012-10-03
|-
| Ubuntu 12.04
| Mihaela
| '''PASS''' - 2012-10-02
| '''PASS''' - 2012-10-02
| '''PASS''' - 2012-10-02
|-
| class="header" | Operating System
| class="header" | Tester
| class="header" | Web Mail
| class="header" | Mac 10.8 Signing
| class="header" | Breakpad
|-
| Windows XP
| Paul/Virgil
| '''PASS''' - 2012-10-02
| '''N/A'''
| '''PASS''' - 2012-10-02
|-
| Windows 7
| Ioana/Paul
| '''PASS''' - 2012-10-02
| '''N/A'''
| '''PASS''' - 2012-10-02
|-
| Windows 8
| Simona
| '''PASS''' - 2012-10-02
| '''N/A'''
| '''PASS''' - 2012-10-02
|-
| Mac OS X
| VladG
| '''PASS''' - 2012-10-03
| '''PASS''' - 2012-10-03
| '''PASS''' - 2012-10-03
|-
| Ubuntu 12.04
| Mihaela
| '''PASS''' - 2012-10-02
| '''N/A'''
| '''PASS''' - 2012-10-02
|}
 
==== Plug-in Compatibility 6 ====
Install the following plug-ins and go to websites which use them, looking for rendering issues, functional issues, performance issues, and crashes.
* Adobe Flash Player
**[http://get.adobe.com/flashplayer/ 11.4 Release]
* [http://www.microsoft.com/silverlight/ Microsoft Silverlight]
* [http://www.real.com/ RealPlayer]
** Real Player plugin crashed when opening pages with content managed by the plugin: [http://www.slp3d2.com/launchsiteglobal/test.cfm click], [http://cf.unc.edu/skillstest/take/realplayer.htm click] - {{bug|667399}}
* [http://www.apple.com/quicktime/download/ QuickTime]
 
==== Add-on Compatibility 6 ====
Test the following websites for rendering issues, functional issues, performance issues, and crashes. Pay particular attention to any problems seen after updating Firefox with add-ons installed.
* [https://addons.mozilla.org/en-US/firefox/compatibility/16.0 Top AMO add-ons]
* AVG toolbar (installed via AVG software)
* Norton toolbar (installed via Norton software)
* [http://sg.toolbar.yahoo.com/ Yahoo toolbar]
* [http://bar.yandex.ru/en/ Yandex toolbar]
* [http://babylon-toolbar.software.informer.com/download/ Babylon toolbar]
* [http://lavasoft.com/ Ad-Aware Antivirus]
 
==== Web Compatibility 6 ====
Test the following websites for rendering issues, functional issues, performance issues, and crashes
# [http://www.alexa.com/topsites Overall Top 10]
# [http://www.alexa.com/topsites/countries Top by Country] (pick a random site from three random countries)
# [http://www.alexa.com/topsites/category Top by Category] (pick a random site from three random categories)
# Webmail clients like gmail, hotmail, and yahoo mail (pay particular attention to any built in web messenger software)
# Try Google's [http://support.google.com/maps/bin/answer.py?hl=en&answer=1630790 MapsGL demo]
# Try YouTube's [http://www.youtube.com/html5 HTML5 demo]
 
==== Web Mail 6 ====
Test the following websites for rendering issues, functional issues, performance issues, and crashes. In particular make sure email, attachments, web messenger, and video chat are working as expected.
# [https://mail.google.com/ Google Mail]
# [https://mail.yahoo.com/ Yahoo Mail]
# [https://www.hotmail.com Hotmail]
# [https://www.outlook.com/ Outlook.com]
 
==== Mac 10.8 Signing 6 ====
NOTE: Test needs to be run from a clean Mac OSX 10.8 install. Please keep a vanilla image handy so you can restore once completing this test ([https://people.mozilla.com/~stmichaud/Gatekeeper/MLGatekeeper.html more info]).
* Download the .dmg file and drag the Firefox.app file to a location on your computer
* Open a terminal window and cd to the folder Firefox.app is located
* Run ''codesign -vv Firefox.app'' in terminal, output should contain ''valid on disk'' and ''satisfies its Designated Requirement''
* Run ''xattr Firefox.app'' in terminal, output should contain ''com.apple.quarantine''
* Double click the Firefox.app icon, you should see a warning about the program being provided by an ''unknown developer'' or having been ''downloaded from the Internet''
* Click "open" and browse to any website, Firefox should start and you should be able to browse
* Quit Firefox and double click the Firefox.app icon again, you should ''not'' be reprompted for the warning in step 5, firefox should start up and browse without any issues
 
Spot check build after this test
* Try all the menu item functions
* Gestures
* Plug-in compatibility
* Top-site compatibility
* Add-on compatibility
* Software update
 
==== Breakpad End-to-End 6 ====
Test the end-to-end functionality in the crash reporting feature, across operating systems.
* Use the [http://ted.mielczarek.org/mozilla/crashme.html Crash Me add-on] to trigger a crash
* Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc
* Repeatedly crash Firefox and submit your reports
* Verify that your session restores properly
* Verify your crash reports are submitted properly by going through about:crashes and clicking the reports
* Try adding comments to your crash report and verify the crash-stats.com report shows the comment
 
=== Update Testing 6 ===
* Partial: 16.0b5
* Complete: 16.0b4 and earlier
 
{|class="querytable" style="width:60%"
|-
| class="header" | Platform
| class="header" | Releasetest: [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-10-02&to=2012-10-03&target=16.0 report]
| class="header" | Beta: [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-10-02&to=2012-10-03&target=16.0 report]
|-
| Windows XP
| '''PASS''' - 2012-10-02
| '''RESULT''' - 2012-10-02
|-
| Windows Vista
| '''PASS''' - 2012-10-02
| '''RESULT''' - 2012-10-02
|-
| Windows 7 32-bit
| '''PASS''' - 2012-10-02
| '''RESULT''' - 2012-10-02
|-
| Windows 7 64-bit
| '''PASS''' - 2012-10-02
| '''RESULT''' - 2012-10-02
|-
| Windows 8 32-bit
| '''manual'''-see below - 2012-10-02
| '''Results''' - yyyy-mm-dd
|-
| Windows 8 64-bit
| '''manual'''-see below - 2012-10-02
| '''Results''' - yyyy-mm-dd
|-
| Mac OS X 10.6
| '''PASS''' - 2012-10-02
| '''RESULT''' - 2012-10-02
|-
| Mac OS X 10.7
| '''PASS''' - 2012-10-02
| '''RESULT''' - 2012-10-02
|-
| Mac OS X 10.8
| '''PASS''' - 2012-10-02
| '''RESULT''' - 2012-10-02
|-
| Ubuntu 12.04 32-bit
| '''PASS''' - 2012-10-02
| '''RESULT''' - 2012-10-02
|-
| Ubuntu 12.04 64-bit
| '''PASS''' - 2012-10-02
| '''RESULT''' - 2012-10-02
|}
 
''Windows 8: manual spotcheck until automation in place''
* RELEASETEST
** '''RESULT''' - Firefox 16.0b5 en-US (partial)
** '''RESULT''' - Firefox 15.0b6 de (complete)
* BETA
** '''RESULT''' - Firefox 16.0b5 en-US (partial)
** '''RESULT''' - Firefox 15.0b6 de (complete)
 
; Issues
 
=== Bug Triage 6 ===
# [http://bit.ly/RquQy9 VERIFYME (10)] : verify the fix and do some exploratory testing around the issue
# [http://bit.ly/Ppe9E0 FIXED (181)] : triage these bugs and test to ensure the fix is effective
# [http://bit.ly/PMaYVx QAWANTED (4)] : triage these bugs and test to fulfill the QA request
# [http://bit.ly/OtEqj1 UNCONFIRMED (18)] : triage these bugs for potential new regressions
 
; Results
* [http://bit.ly/MXYdqS VERIFIED]: 87/303
* [http://bit.ly/POyxuA WONTVERIFY]: 31/303
* [http://bit.ly/O09WSK HELPWANTED]: 4/303
 
== Release Candidate ==
{| class="querytable" style="width:30%"
| style="width:10%" |
| style="width:20%" |
|-
| '''Builds'''
| [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0-candidates/build1/ HTTP]
|-
| '''ID'''
| 20121005155445
|-
| '''Changeset'''
| [http://hg.mozilla.org/releases/mozilla-release/rev/10fe550fadc6 10fe550fadc6]
|}
 
; Checklist
* {{done|}} [[Releases/Firefox_16/Test_Plan#Automated_Testing_RC|Automated Testing]]
* {{done|}} [[Releases/Firefox_16/Test_Plan#Manual_Testing_RC|Manual Testing]]
* {{done|}} [[Releases/Firefox_16/Test_Plan#Update_Testing_RC|Update Testing]]
* {{ok|}} [[Releases/Firefox_16/Test_Plan#Bug_Triage_RC|Bug Triage]]
* {{|}} Peer Sign-off
* {{ | }} Final sign-off sent and updates qualified on Release channel
 
=== Automated Testing RC===
The following are the detailed results of the automated Mozmill test runs.
{|class="querytable" style="width:50%"
|-
| class="header" style="width:20%" | Platform
| class="header" style="width:30%" | RESULT: [http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-10-05&to=2012-10-09 report]
|-
| Windows XP
| '''PASS''' - 2012-10-08
|-
| Windows Vista
| '''PASS''' - 2012-10-08
  |-
  |-
  | Windows 7 32-bit
  | Windows 7 32-bit
| '''PASS''' - 2012-10-08
|-
| Windows 7 64-bit
| '''PASS''' - 2012-10-08
|-
| Windows 8 32-bit
| '''PASS''' - 2012-10-08 
|-
| Windows 8 64-bit
| '''PASS''' - 2012-10-08 
|-
| Mac OS X 10.6
| '''PASS''' - 2012-10-08
|-
| Mac OS X 10.7
| '''PASS''' - 2012-10-08
|-
| Mac OS X 10.8
| '''PASS''' - 2012-10-08
|-
| Ubuntu 12.04 32-bit
| '''PASS''' - 2012-10-08
|-
| Ubuntu 12.04 64-bit
| '''PASS''' - 2012-10-08
|}
; Test Failures ([https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA known bugs])
=== Manual Testing RC ===
{| class="querytable"
| class="header" | Operating System
| class="header" | Tester
| class="header" | Plug-in Compatibility
| class="header" | Add-on Compatibility
| class="header" | Web Compatibility
|-
| Windows XP
  | Ioana
  | Ioana
  | '''Results''' - 2012-07-19
  | '''PASS''' - 2012-10-09
| '''PASS''' - 2012-10-09
| '''PASS''' - 2012-10-09
|-
| Windows 7
| Simona
| '''PASS''' - 2012-10-08
| '''PASS''' - 2012-10-09
| '''PASS''' - 2012-10-09
|-
| Windows 8
| Mihaela
| '''PASS''' - 2012-10-09
| '''PASS''' - 2012-10-09
| '''PASS''' - 2012-10-09
|-
| Mac OS X
| VladG
| '''PASS''' - 2012-10-09
| '''PASS''' - 2012-10-09
| '''PASS''' - 2012-10-09
|-
| Ubuntu 12.04
| Virgil
| '''PASS''' - 2012-10-08
| '''PASS''' - 2012-10-09
| '''PASS''' - 2012-10-08
|-
| class="header" | Operating System
| class="header" | Tester
| class="header" | Web Mail
| class="header" | Mac 10.8 Signing
| class="header" | Breakpad
|-
| Windows XP
| Paul
| '''PASS''' - 2012-10-09
| '''N/A'''
| '''PASS''' - 2012-10-09
|-
| Windows 7
| Simona
| '''PASS''' - 2012-10-09
| '''N/A'''
| '''PASS''' - 2012-10-09
|-
| Windows 8
| Mihaela
| '''PASS''' - 2012-10-09
| '''N/A'''
| '''PASS''' - 2012-10-09
|-
| Mac OS X
| VladG
| '''PASS''' - 2012-10-09
| '''PASS''' - 2012-10-08
| '''PASS''' - 2012-10-09
|-
| Ubuntu 12.04
| Virgil
| '''PASS''' - 2012-10-08
| '''N/A'''
| '''PASS''' - 2012-10-09
|}
 
==== Plug-in Compatibility ====
Install the following plug-ins and go to websites which use them, looking for rendering issues, functional issues, performance issues, and crashes.
* Adobe Flash Player
**[http://get.adobe.com/flashplayer/ 11.4 Release]
* [http://www.microsoft.com/silverlight/ Microsoft Silverlight]
* [http://www.real.com/ RealPlayer]
** Real Player plugin crashed when opening pages with content managed by the plugin: [http://www.slp3d2.com/launchsiteglobal/test.cfm click], [http://cf.unc.edu/skillstest/take/realplayer.htm click] - {{bug|667399}}
* [http://www.apple.com/quicktime/download/ QuickTime]
 
==== Add-on Compatibility ====
Test the following websites for rendering issues, functional issues, performance issues, and crashes. Pay particular attention to any problems seen after updating Firefox with add-ons installed.
* [https://addons.mozilla.org/en-US/firefox/compatibility/16.0 Top AMO add-ons]
* AVG toolbar (installed via AVG software)
* Norton toolbar (installed via Norton software)
* [http://sg.toolbar.yahoo.com/ Yahoo toolbar]
* [http://bar.yandex.ru/en/ Yandex toolbar]
* [http://babylon-toolbar.software.informer.com/download/ Babylon toolbar]
* [http://lavasoft.com/ Ad-Aware Antivirus]
 
==== Web Compatibility ====
Test the following websites for rendering issues, functional issues, performance issues, and crashes
# [http://www.alexa.com/topsites Overall Top 10]
# [http://www.alexa.com/topsites/countries Top by Country] (pick a random site from three random countries)
# [http://www.alexa.com/topsites/category Top by Category] (pick a random site from three random categories)
# Webmail clients like gmail, hotmail, and yahoo mail (pay particular attention to any built in web messenger software)
# Try Google's [http://support.google.com/maps/bin/answer.py?hl=en&answer=1630790 MapsGL demo]
# Try YouTube's [http://www.youtube.com/html5 HTML5 demo]
 
==== Web Mail ====
Test the following websites for rendering issues, functional issues, performance issues, and crashes. In particular make sure email, attachments, web messenger, and video chat are working as expected.
# [https://mail.google.com/ Google Mail]
# [https://mail.yahoo.com/ Yahoo Mail]
# [https://www.hotmail.com Hotmail]
# [https://www.outlook.com/ Outlook.com]
 
==== Mac 10.8 Signing ====
NOTE: Test needs to be run from a clean Mac OSX 10.8 install. Please keep a vanilla image handy so you can restore once completing this test ([https://people.mozilla.com/~stmichaud/Gatekeeper/MLGatekeeper.html more info]).
* Download the .dmg file and drag the Firefox.app file to a location on your computer
* Open a terminal window and cd to the folder Firefox.app is located
* Run ''codesign -vv Firefox.app'' in terminal, output should contain ''valid on disk'' and ''satisfies its Designated Requirement''
* Run ''xattr Firefox.app'' in terminal, output should contain ''com.apple.quarantine''
* Double click the Firefox.app icon, you should see a warning about the program being provided by an ''unknown developer'' or having been ''downloaded from the Internet''
* Click "open" and browse to any website, Firefox should start and you should be able to browse
* Quit Firefox and double click the Firefox.app icon again, you should ''not'' be reprompted for the warning in step 5, firefox should start up and browse without any issues
 
Spot check build after this test
* Try all the menu item functions
* Gestures
* Plug-in compatibility
* Top-site compatibility
* Add-on compatibility
* Software update
 
==== Breakpad End-to-End ====
Test the end-to-end functionality in the crash reporting feature, across operating systems.
* Use the [http://ted.mielczarek.org/mozilla/crashme.html Crash Me add-on] to trigger a crash
* Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc
* Repeatedly crash Firefox and submit your reports
* Verify that your session restores properly
* Verify your crash reports are submitted properly by going through about:crashes and clicking the reports
* Try adding comments to your crash report and verify the crash-stats.com report shows the comment
 
=== Update Testing RC ===
* Partial: 15.0.1
* Complete: 12.0 and earlier
 
{|class="querytable" style="width:60%"
|-
| class="header" | Platform
| class="header" | Releasetest: [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-10-08&to=2012-10-10&target=16.0 report]
| class="header" | Beta: [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-10-08&to=2012-10-10&target=16.0 report]
|-
| Windows XP
| '''PASS''' - 2010-10-08
| '''RESULTS''' - yyyy-mm-dd
|-
| Windows Vista
| '''PASS''' - 2010-10-08
| '''RESULTS''' - yyyy-mm-dd
|-
| Windows 7 32-bit
| '''PASS''' - 2010-10-08
| '''RESULTS''' - yyyy-mm-dd
  |-
  |-
  | Windows 7 64-bit
  | Windows 7 64-bit
| '''PASS''' - 2010-10-08
| '''RESULTS''' - yyyy-mm-dd
|-
| Windows 8 32-bit
| '''PASS'''- see below - 2012-10-09
| '''RESULTS''' - yyyy-mm-dd
|-
| Windows 8 64-bit
| '''PASS'''- see below - 2012-10-08
| '''RESULTS''' - yyyy-mm-dd
|-
| Mac OS X 10.6
| '''PASS''' - 2010-10-08
| '''RESULTS''' - yyyy-mm-dd
|-
| Mac OS X 10.7
| '''PASS''' - 2010-10-08
| '''RESULTS''' - yyyy-mm-dd
|-
| Mac OS X 10.8
| '''PASS''' - 2010-10-08
| '''RESULTS''' - yyyy-mm-dd
|-
| Ubuntu 12.04 32-bit
| '''PASS''' - 2010-10-08
| '''RESULTS''' - yyyy-mm-dd
|-
| Ubuntu 12.04 64-bit
| '''PASS''' - 2010-10-08
| '''RESULTS''' - yyyy-mm-dd
|}
''Windows 8: manual spotcheck until automation in place''
* BETATEST
** '''PASS''' - Firefox 15.0.1 en-US de (partial)
** '''PASS''' - Firefox 12.0 en-US fr (complete)
* RELEASE
** '''RESULT''' - Firefox 15.0.1 en-US de (partial)
** '''RESULT''' - Firefox 12.0 en-US de (complete)
; Issues
=== Bug Triage RC ===
# [http://bit.ly/RquQy9 VERIFYME (9)] : verify the fix and do some exploratory testing around the issue
# [http://bit.ly/Ppe9E0 FIXED (178)] : triage these bugs and test to ensure the fix is effective
# [http://bit.ly/PMaYVx QAWANTED (8)] : triage these bugs and test to fulfill the QA request
# [http://bit.ly/OtEqj1 UNCONFIRMED (21)] : triage these bugs for potential new regressions
; Results
* [http://bit.ly/MXYdqS VERIFIED]: 89/304
* [http://bit.ly/POyxuA WONTVERIFY]: 32/304
* [http://bit.ly/O09WSK HELPWANTED]: 5/304
== 16.0.1 ==
{| class="querytable" style="width:30%"
| style="width:10%" |
| style="width:20%" |
|-
| '''Builds'''
| [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0.1-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0.1-candidates/build1/ HTTP]
|-
| '''ID'''
| 20121010144125
|-
| '''Changeset'''
| [https://hg.mozilla.org/releases/mozilla-release/rev/7789ab68a99d 7789ab68a99d] ([http://hg.mozilla.org/releases/mozilla-release/pushloghtml?fromchange=10fe550fadc6&tochange=7789ab68a99d pushlog])
|}
; Checklist
* {{done|}} [[Releases/Firefox_16/Test_Plan#Automated_Testing_7|Automated Testing]]
* {{done|}} [[Releases/Firefox_16/Test_Plan#Manual_Testing_7|Manual Testing]]
* {{done|}} [[Releases/Firefox_16/Test_Plan#Update_Testing_7|Update Testing]]
* {{ | }} Final sign-off sent and updates qualified on Release channel
=== Automated Testing 7 ===
The following are the detailed results of the automated Mozmill test runs.
{|class="querytable" style="width:50%"
|-
| class="header" style="width:20%" | Platform
| class="header" style="width:30%" | RESULT: [http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-10-10&to=2012-10-11 report]
|-
| Windows XP
| '''PASS''' - 2012-10-10
|-
| Windows Vista
| '''PASS''' - 2012-10-10
|-
| Windows 7 32-bit
| '''PASS''' - 2012-10-10
|-
| Windows 7 64-bit
| '''PASS''' - 2012-10-10
|-
| Windows 8 32-bit
| '''PASS''' - 2012-10-10
|-
| Windows 8 64-bit
| '''PASS''' - 2012-10-10
|-
| Mac OS X 10.6
| '''PASS''' - 2012-10-10
|-
| Mac OS X 10.7
| '''PASS''' - 2012-10-10
|-
| Mac OS X 10.8
| '''PASS''' - 2012-10-10
|-
| Ubuntu 12.04 32-bit
| '''PASS''' - 2012-10-10
|-
| Ubuntu 12.04 64-bit
| '''PASS''' - 2012-10-10
|}
; Test Failures ([https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA known bugs])
=== Manual Testing 7 ===
There will be four bug verifications and testing around these bugs
; Results
{| class="querytable"
| class="header" | Platform
| class="header" | Tester
| class="header" | 16.0.1 Candidates
|-
| Windows XP
| juanb, ashughes, mwobensmith
| '''PASS''' - 2012-10-10
|-
| Windows Vista
| tester
| '''RESULT''' - 2012-10-10
|-
| Windows 7 32-bit
  | tester
  | tester
  | '''Results''' - 2012-07-19
  | '''RESULT''' - 2012-10-10
|-
| Windows 7 64-bit
| juanb, ashughes, mwobensmith
| '''PASS''' - 2012-10-10
  |-
  |-
  | Mac OS X 10.6
  | Mac OS X 10.6
  | tester
  | tester
  | '''Results''' - 2012-07-19
  | '''RESULT''' - 2012-10-10
  |-
  |-
  | Mac OS X 10.7.4
  | Mac OS X 10.7
| juanb, ashughes, mwobensmith
| '''PASS''' - 2012-10-10
|-
| Mac OS X 10.8
  | tester
  | tester
  | '''Results''' - 2012-07-19
  | '''RESULT''' - 2012-10-10
  |-
  |-
  | Ubuntu 12.04 32-bit
  | Ubuntu 12.04 32-bit
  | tester
  | tester
  | '''Results''' - 2012-07-19
  | '''RESULT''' - 2012-10-10
  |-
  |-
  | Ubuntu 12.04 64-bit
  | Ubuntu 12.04 64-bit
  | tester
  | juanb, ashughes, mwobensmith
  | '''Results''' - 2012-07-19
| '''PASS''' - 2012-10-10
|}
 
; Issues Found
 
==== Breakpad End-to-End 7 ====
Test the end-to-end functionality in the crash reporting feature, across operating systems.
* Use the [http://ted.mielczarek.org/mozilla/crashme.html Crash Me add-on] to trigger a crash
* Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc
* Repeatedly crash Firefox and submit your reports
* Verify that your session restores properly
* Verify your crash reports are submitted properly by going through about:crashes and clicking the reports
* Try adding comments to your crash report and verify the crash-stats.com report shows the comment
 
=== Update Testing 7 ===
* Partial: 13.0.1, 14.0.1, 15.0.1
* Complete: 12.0, 13.0
 
{|class="querytable" style="width:60%"
|-
| class="header" | Platform
| class="header" | Releasetest: [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-10-10&to=2012-10-10&target=16.0 report]
| class="header" | Release: [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-10-11&to=2012-10-11&target=16.0 report]
|-
| Windows XP
| '''PASS''' - 2012-10-11
| '''PASS''' - 2012-10-11
|-
| Windows Vista
| '''PASS''' - 2012-10-11
| '''PASS''' - 2012-10-11
|-
| Windows 7 32-bit
| '''PASS''' - 2012-10-11
| '''PASS''' - 2012-10-11
|-
| Windows 7 64-bit
| '''PASS''' - 2012-10-11
| '''PASS''' - 2012-10-11
|-
| Windows 8 32-bit
| '''Pass'''- juanb - 2012-10-11
| '''Pass''' - juanb - 2012-10-11
|-
| Windows 8 64-bit
| '''Pass'''- juanb - 2012-10-11
| '''Pass''' - 2012-10-11
|-
| Mac OS X 10.6
| '''PASS''' - 2012-10-11
| '''PASS''' - 2012-10-11
|-
| Mac OS X 10.7
| '''PASS''' - 2012-10-11
| '''PASS''' - 2012-10-11
|-
| Mac OS X 10.8
| '''PASS''' - 2012-10-11
| '''PASS''' - 2012-10-11
|-
| Ubuntu 12.04 32-bit
| '''PASS''' - 2012-10-11
| '''PASS''' - 2012-10-11
|-
| Ubuntu 12.04 64-bit
| '''PASS''' - 2012-10-11
| '''PASS''' - 2012-10-11
|}
 
''Windows 8: manual spotcheck until automation in place''
* RELEASETEST
** '''PASS''' - Firefox 16.0 en-US (partial)
** '''PASS''' - Firefox 15.0.1 de (partial)
* RELEASE
** '''PASS''' - Firefox 16.0 en-US (partial)
** '''PASS''' - Firefox 12.0 fr (complete)
 
; Issues
 
== 16.0.2 ==
{| class="querytable" style="width:30%"
| style="width:10%" |
| style="width:20%" |
|-
| '''Builds'''
| [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0.2-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0.2-candidates/build1/ HTTP]
|-
| '''ID'''
| 20121024073032
|-
| '''Changeset'''
| [https://hg.mozilla.org/releases/mozilla-release/rev/3995ff235342 3995ff235342] ([http://hg.mozilla.org/releases/mozilla-release/pushloghtml?fromchange=7789ab68a99d&tochange=3995ff235342 pushlog])
|}
 
; Checklist
* {{done|}} [[Releases/Firefox_16/Test_Plan#Automated_Testing_8|Automated Testing]]
* {{done|}} [[Releases/Firefox_16/Test_Plan#Manual_Testing_8|Manual Testing]]
* {{ok|}} [[Releases/Firefox_16/Test_Plan#Update_Testing_8|Update Testing]]
* {{ | }} Final sign-off sent and updates qualified on Beta channel
 
=== Automated Testing 8 ===
The following are the detailed results of the automated Mozmill test runs.
{|class="querytable" style="width:50%"
|-
| class="header" style="width:20%" | Platform
| class="header" style="width:30%" | RESULT: [http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-10-24&to=2012-10-24 report]
|-
| Windows XP
| '''PASS''' - 2012-10-24
|-
| Windows Vista
| '''PASS''' - 2012-10-24
|-
| Windows 7 32-bit
| '''PASS''' - 2012-10-24
|-
| Windows 7 64-bit
| '''PASS''' - 2012-10-24
|-
| Windows 8 32-bit
| '''PASS''' - 2012-10-24
|-
| Windows 8 64-bit
| '''PASS''' - 2012-10-24
|-
| Mac OS X 10.6
| '''PASS''' - 2012-10-24
|-
| Mac OS X 10.7
| '''PASS''' - 2012-10-24
|-
| Mac OS X 10.8
| '''PASS''' - 2012-10-24
|-
| Ubuntu 12.04 32-bit
| '''PASS''' - 2012-10-24
|-
| Ubuntu 12.04 64-bit
| '''PASS''' - 2012-10-24
|}
 
; Test Failures ([https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA known bugs])
 
=== Manual Testing 8 ===
There will be X bug verifications and testing around these bugs. Basic functionality spot checking - [https://mozqa.etherpad.mozilla.org/Firefox16-0-2 etherpad].
 
; Results
{| class="querytable"
| class="header" | Platform
| class="header" | Tester
| class="header" | 16.0.2 Candidates
|-
| Windows XP
| Simona
| '''PASS''' - 2012-10-25
|-
| Windows Vista
| Simona
| '''PASS''' - 2012-10-25
|-
| Windows 7 32-bit
| Mihaela
| '''PASS''' - 2012-10-25
|-
| Windows 7 64-bit
| Manuela
| '''PASS''' - 2012-10-25
|-
| Mac OS X 10.6
| Virgil
| '''PASS''' - 2012-10-25
|-
| Mac OS X 10.7
| Virgil
| '''PASS''' - 2012-10-25
|-
| Mac OS X 10.8
| Mihaela
| '''PASS''' - 2012-10-25
|-
| Ubuntu 12.04 32-bit
| Ioana
| '''PASS''' - 2012-10-25
|-
| Ubuntu 12.04 64-bit
| Ioana
| '''PASS''' - 2012-10-25
|}
 
; Issues Found
 
==== Breakpad End-to-End 8 ====
Test the end-to-end functionality in the crash reporting feature, across operating systems.
* Use the [http://ted.mielczarek.org/mozilla/crashme.html Crash Me add-on] to trigger a crash
* Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc
* Repeatedly crash Firefox and submit your reports
* Verify that your session restores properly
* Verify your crash reports are submitted properly by going through about:crashes and clicking the reports
* Try adding comments to your crash report and verify the crash-stats.com report shows the comment
 
=== Update Testing 8 ===
* Partial: 14.0.1, 15.0.1, 16.0.1
* Complete: 12.0, 13.0
 
{|class="querytable" style="width:60%"
|-
| class="header" | Platform
| class="header" | Releasetest: [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-10-24&to=2012-10-25&target=16.0 report]
| class="header" | Beta: [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-10-24&to=2012-10-25&target=16.0 report]
|-
| Windows XP
| '''RESULT''' - 2012-10-24
| '''RESULT''' - 2012-10-24
|-
| Windows Vista
| '''RESULT''' - 2012-10-24
| '''RESULT''' - 2012-10-24
|-
| Windows 7 32-bit
| '''RESULT''' - 2012-10-24
| '''RESULT''' - 2012-10-24
|-
| Windows 7 64-bit
| '''RESULT''' - 2012-10-24
| '''RESULT''' - 2012-10-24
|-
| Windows 8 32-bit
| '''manual'''-see below - 2012-10-02
| '''Results''' - yyyy-mm-dd
|-
| Windows 8 64-bit
| '''manual'''-see below - 2012-10-02
  | '''Results''' - yyyy-mm-dd
|-
| Mac OS X 10.6
| '''RESULT''' - 2012-10-24
| '''RESULT''' - 2012-10-24
|-
| Mac OS X 10.7
| '''RESULT''' - 2012-10-24
| '''RESULT''' - 2012-10-24
|-
| Mac OS X 10.8
| '''RESULT''' - 2012-10-24
| '''RESULT''' - 2012-10-24
|-
| Ubuntu 12.04 32-bit
| '''RESULT''' - 2012-10-24
| '''RESULT''' - 2012-10-24
  |-
  |-
| Ubuntu 12.04 64-bit
| '''RESULT''' - 2012-10-24
| '''RESULT''' - 2012-10-24
  |}
  |}
''Windows 8: manual spotcheck until automation in place''
* RELEASETEST
** '''RESULT''' - Firefox 16.0.1 en-US (partial)
** '''RESULT''' - Firefox 13.0 de (complete)
* RELEASE
** '''RESULT''' - Firefox 16.0.1 en-US (partial)
** '''RESULT''' - Firefox 12.0 fr (complete)
; Issues
== Issues ==
=== Blockers ===
<bugzilla>
{
"id": "793126"
}
</bugzilla>
=== Non-Blockers ===
<bugzilla>
{
"id": "786655, 786970, 786972, 787056, 789463,797332"
}
</bugzilla>
=== Previously Known ===
<bugzilla>
{
"id": "768375, 686782, 667399, 785754, 779492, 775082, 777051, 747909, 787056, 675645, 778819, 786655, 781190, 78414, 788423, 786237, 791195, 717178, 794924, 692446, 792422, 797116"
}
</bugzilla>
Simona Badau
Confirmed users
1,333

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/452401...484320"