Releases/Firefox 16/Test Plan: Difference between revisions
Line 1: | Line 1: | ||
<small> | <p><small><a href="Releases/Firefox 15/Test Plan">« Firefox 15</a> | <a href="Releases/Firefox_16/Test_Plan" class="selflink">Firefox 16</a> | <a href="Releases/Firefox 17/Test Plan">Firefox 17 »</a> </small> | ||
</p> | |||
<h1> Firefox 16 Test Plan </h1> | |||
<h2> Summary </h2> | |||
<table class="fullwidth-table"> | |||
<tr> | |||
<td style="width:15%"> <b>Milestone</b> | |||
</td> | |||
<td> <a href="https://wiki.mozilla.org/Firefox/Flight_Tracking#Firefox_16">Firefox 16</a> | |||
</td></tr> | |||
<tr> | |||
<td> <b>Status</b> | |||
</td> | |||
<td> Currently in <b>Beta</b> -- merging to Release on October 8, 2012 | |||
</td></tr> | |||
<tr> | |||
<td> <b>Lead</b> | |||
</td> | |||
<td> <a href="mailto:jbecerra@mozilla.com">Juan Becerra</a> (irc: juanb) | |||
</td></tr> | |||
<tr> | |||
<td> <b>2nd</b> | |||
</td> | |||
<td> <a href="mailto:ahughes@mozilla.com">Anthony Hughes</a> (irc: ashughes) | |||
</td></tr> | |||
<tr> | |||
<td> <b>Builds</b> | |||
</td> | |||
<td> <a href="https://www.mozilla.org/beta/">beta.mozilla.org</a> | |||
</td></tr></table> | |||
<p>The following is the test plan for Firefox 16 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 16. After the release this document will be used as an archive of what was done to validate this release. | |||
</p><p><i>If you have some free time, please pick a task below and get in touch with one of the leads.</i> | |||
</p> | |||
<h2> Schedule </h2> | |||
<p>The following is a description of the tasks which need to be completed throughout this release. | |||
</p> | |||
<ul><li> 2012-06-04: <span class="fck_mw_template">{{done|}}</span> Nightly Merge | |||
</li><li> 2012-07-17: <span class="fck_mw_template">{{done|}}</span> Aurora Merge | |||
</li><li> 2012-08-28: <span class="fck_mw_template">{{done|}}</span> Pre-Beta feature sign-offs | |||
</li><li> 2012-08-30: <span class="fck_mw_template">{{done|}}</span> Beta 1 | |||
</li><li> 2012-09-06: <span class="fck_mw_template">{{done|}}</span> Beta 2 | |||
</li><li> 2012-09-13: <span class="fck_mw_template">{{ok|}}</span> Beta 3 | |||
</li><li> 2012-09-20: <span class="fck_mw_template">{{|}}</span> Beta 4 | |||
</li><li> 2012-09-27: <span class="fck_mw_template">{{|}}</span> Beta 5 | |||
</li><li> 2012-10-04: <span class="fck_mw_template">{{|}}</span> Beta 6 | |||
</li><li> 2012-10-06: <span class="fck_mw_template">{{|}}</span> Release Candidate | |||
</li><li> 2012-10-09: <span class="fck_mw_template">{{|}}</span> Final | |||
</li></ul> | |||
<h2> Nightly </h2> | |||
<h3> Merge </h3> | |||
<dl><dt> Functional Automation - result | |||
</dt></dl> | |||
<dl><dt> Update Spotchecks - result | |||
</dt></dl> | |||
<h3> Features </h3> | |||
<p><i>See the <a href="https://wiki.mozilla.org/Features/Release_Tracking">Feature Release Tracking</a> wiki for a general overview</i> | |||
</p> | |||
<table class="querytable"> | |||
<tr> | |||
<td class="header"> Feature | |||
</td> | |||
<td class="header"> Owner | |||
</td> | |||
<td class="header"> Status | |||
</td> | |||
<td class="header"> Tests | |||
</td></tr> | |||
<tr> | |||
<td> <a href="Platform/Features/Incremental GC">Incremental GC</a> | |||
</td> | |||
<td> Ioana Budnar | |||
</td> | |||
<td> On track | |||
</td> | |||
<td> | |||
</td></tr> | |||
<tr> | |||
<td> <a href="Opt-in activation for plugins">Opt-in activation for plugins</a> | |||
</td> | |||
<td> Paul Silaghi | |||
</td> | |||
<td> On track | |||
</td> | |||
<td> | |||
</td></tr> | |||
<tr> | |||
<td> <a href="DevTools/Features/DeveloperToolbar">Developer Toolbar</a> | |||
</td> | |||
<td> Virgil Dicu | |||
</td> | |||
<td> On track | |||
</td> | |||
<td> | |||
</td></tr> | |||
<tr> | |||
<td> <a href="DevTools/Features/GCLI">Command Line</a> | |||
</td> | |||
<td> Mihaela Velimiroviciu | |||
</td> | |||
<td> On track | |||
</td> | |||
<td> | |||
</td></tr> | |||
<tr> | |||
<td> <a href="Show PDF inline">Show PDF inline</a> | |||
</td> | |||
<td> Mihaela Velimiroviciu | |||
</td> | |||
<td> Disabled until Fx18(<span class="fck_mw_template">{{bug|786012}}</span>) | |||
</td> | |||
<td> | |||
</td></tr> | |||
<tr> | |||
<td> <a href="User:P.A./Panel-based Download Manager">Panel-based Download Manager</a> | |||
</td> | |||
<td> Simona Badau | |||
</td> | |||
<td> Delayed to Fx18 | |||
</td> | |||
<td> | |||
</td></tr> | |||
<tr> | |||
<td> <a href="OS X 10.7 support">OS X 10.7 support</a> | |||
</td> | |||
<td> Vlad Ghetiu | |||
</td> | |||
<td> Not implemented | |||
</td> | |||
<td> | |||
</td></tr> | |||
</table> | |||
<h3> Endurance Automation </h3> | |||
<p>The purpose of this section is to track any potential memory regressions via spikes reported in the dashboard. | |||
</p> | |||
<dl><dt> What to Do | |||
</dt></dl> | |||
<ol><li> Check the <a href="http://mozmill-ci.blargon7.com/#/endurance/charts?platform=All&from=2012-06-05&to=2012-07-17">Endurance Charts</a> | |||
</li><li> If you see an anomalous event (ie. spike or dip), get the report for the specific event | |||
</li><li> If the event happens twice in a row, file a bug under Mozmill Tests | |||
</li><li> Notify <a href="mailto:ahughes@mozilla.com">Anthony Hughes</a> of the issue | |||
</li><li> Record the event in this wiki below | |||
</li></ol> | |||
<dl><dt> Current Status (Mihaela Velimiroviciu) - Last Updated Jul 18th, 2012 | |||
</dt></dl> | |||
<table class="bugzilla ui-helper-reset"> | |||
<thead> | |||
<tr> | |||
<th>ID</th> | |||
<th>Summary</th> | |||
<th>Status</th> | |||
<th>Priority</th> | |||
</tr> | |||
</thead> | |||
<tbody> | |||
<tr> | |||
<td><a href="https://bugzilla.mozilla.org/show_bug.cgi?id=773632">773632</a></td> | |||
<td><a href="https://bugzilla.mozilla.org/show_bug.cgi?id=773632">High results on mozmill endurance test on Mac 10.7.4_x64 since Jul 5</a></td> | |||
<td>NEW</td> | |||
<td>--</td> | |||
</tr> | |||
<tr> | |||
<td><a href="https://bugzilla.mozilla.org/show_bug.cgi?id=764823">764823</a></td> | |||
<td><a href="https://bugzilla.mozilla.org/show_bug.cgi?id=764823">Memory spike discovered by endurance tests on Nightly Linux and Mac between June 12 and 13</a></td> | |||
<td>RESOLVED</td> | |||
<td>--</td> | |||
</tr> | |||
<tr> | |||
<td><a href="https://bugzilla.mozilla.org/show_bug.cgi?id=767348">767348</a></td> | |||
<td><a href="https://bugzilla.mozilla.org/show_bug.cgi?id=767348">Memory consumption shows an ascending trend on Nightly/Win 6.1 x64</a></td> | |||
<td>RESOLVED</td> | |||
<td>--</td> | |||
</tr> | |||
</tbody> | |||
</table> | |||
<p><br /> | |||
</p> | |||
<h3> Functional Automation </h3> | |||
<p>The purpose of this section is to track any potential functional regressions via results reported in the dashboard. | |||
</p> | |||
<dl><dt> What to Do | |||
</dt></dl> | |||
<ol><li> Check the <a href="http://mozmill-release.blargon7.com/#/functional/reports?branch=13.0&platform=All&from=2012-06-04&to=2012-07-17">testrun reports</a> | |||
</li><li> If you see any failing tests, check that there are not any <a href="http://bit.ly/wZL8zF">bugs on file</a> for it already | |||
</li><li> If not, file a bug under Mozmill Tests | |||
</li><li> Notify <a href="mailto:ahughes@mozilla.com">Anthony Hughes</a> of the issue | |||
</li><li> Record the event in this wiki below | |||
</li></ol> | |||
<dl><dt> Current Status (Vlad Maniac / Alex Lakatos) - Last Updated | |||
</dt></dl> | |||
<p><br /> | |||
</p> | |||
<h3> Bug Triage </h3> | |||
<p>The purpose of this section is to triage UNCONFIRMED incoming bugs. | |||
</p> | |||
<dl><dt> What to Do | |||
</dt></dl> | |||
<ol><li> Check the <a href="http://bit.ly/HXTYdN">UNCONFIRMED</a> bugs | |||
</li><li> Test the bugs in the latest Firefox Nightly | |||
</li><li> If you can't reproduce the bug, ask for the reporter to provide more information | |||
</li><li> If you reproduce the bug, add your testing information and mark the bug NEW | |||
</li><li> Notify <a href="mailto:ahughes@mozilla.com">Anthony Hughes</a> of the NEW issue | |||
</li></ol> | |||
<dl><dt> Current Status (Vlad Ghetiu / Virgil Dicu) - Last Updated | |||
</dt></dl> | |||
<p><a href="http://bit.ly/z2MmKE">Result Spreadsheet</a> | |||
</p><p><br /> | |||
</p> | |||
<h2> Aurora Merge </h2> | |||
<p>The following are the detailed results of the automated BFT test runs. | |||
</p> | |||
<table class="querytable" style="width:50%"> | |||
<tr> | |||
<td class="header"> Platform | |||
</td> | |||
<td class="header"> Result: <a href="http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-07-18&to=2012-07-20">report</a> | |||
</td></tr> | |||
<tr> | |||
<td> Windows 2000 | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-19 | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-19 | |||
</td></tr> | |||
<tr> | |||
<td> Windows Vista | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-19 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-19 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-19 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.6 | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-19 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.7 | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-19 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 11.04 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-19 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 11.04 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-19 | |||
</td></tr></table> | |||
<dl><dt>Known Issues | |||
</dt></dl> | |||
<table class="bugzilla ui-helper-reset"> | |||
<thead> | |||
<tr> | |||
<th>ID</th> | |||
<th>Summary</th> | |||
<th>Status</th> | |||
<th>Priority</th> | |||
</tr> | |||
</thead> | |||
<tbody> | |||
<tr> | |||
<td><a href="https://bugzilla.mozilla.org/show_bug.cgi?id=770117">770117</a></td> | |||
<td><a href="https://bugzilla.mozilla.org/show_bug.cgi?id=770117">Test failure "Number of visible rows should equal max rows - '1' should equal '6'" in /testAwesomeBar/testVisibleItemsMax.js</a></td> | |||
<td>RESOLVED</td> | |||
<td>P2</td> | |||
</tr> | |||
<tr> | |||
<td><a href="https://bugzilla.mozilla.org/show_bug.cgi?id=719982">719982</a></td> | |||
<td><a href="https://bugzilla.mozilla.org/show_bug.cgi?id=719982">Failure in testAddons_RestartlessExtensionWorksAfterRestart | Modal dialog has been found and processed</a></td> | |||
<td>RESOLVED</td> | |||
<td>P2</td> | |||
</tr> | |||
</tbody> | |||
</table> | |||
<p><br /> | |||
</p> | |||
<h3> Updates </h3> | |||
<p>The following are details of update verifications for Firefox 15 merge to Aurora | |||
</p> | |||
<dl><dt> Auroratest | |||
</dt></dl> | |||
<ul><li> <span class="fck_mw_template">{{|}}</span> [tester] Windows XP: | |||
</li><li> <span class="fck_mw_template">{{|}}</span> [tester] Windows 7: | |||
</li><li> <span class="fck_mw_template">{{|}}</span> [tester] Mac OS X 10.6: | |||
</li><li> <span class="fck_mw_template">{{|}}</span> [tester] Ubuntu 11.04: | |||
</li></ul> | |||
<dl><dt> Aurora | |||
</dt></dl> | |||
<ul><li> <span class="fck_mw_template">{{|}}</span> [tester] Windows XP: | |||
</li><li> <span class="fck_mw_template">{{|}}</span> [tester] Windows 7: | |||
</li><li> <span class="fck_mw_template">{{|}}</span> [tester] Mac OS X 10.6: | |||
</li><li> <span class="fck_mw_template">{{|}}</span> [tester] Ubuntu 11.04: | |||
</li></ul> | |||
<dl><dt> Issues Found | |||
</dt></dl> | |||
<h3> Manual Spot Checks and Feature Sign-Off </h3> | |||
<p>Are all features implemented at this point working? | |||
</p><p>Manual spot Checks [spreadsheet] | |||
</p> | |||
<ul><li>Platforms: Windows XP, Windows 7 x64, Ubuntu 11.10, Mac 10.6 | |||
</li><li>Found known issues: | |||
</li><li>New issues: | |||
</li><li>Status: | |||
</li></ul> | |||
<p>Run the <b>Firefox 16 Aurora Merge Feature Sign-Off</b> tests in <a href="https://moztrap.mozilla.org/runtests/">MozTrap</a> and report results below. | |||
</p> | |||
<table class="querytable"> | |||
<tr> | |||
<td class="header"> Feature | |||
</td> | |||
<td class="header"> Owner | |||
</td> | |||
<td class="header"> Win32 | |||
</td> | |||
<td class="header"> Mac | |||
</td> | |||
<td class="header"> Linux | |||
</td></tr> | |||
<tr> | |||
<td><a href="https://wiki.mozilla.org/User:P.A./Panel-based_Download_Manager">Panel Based Download Manager</a> | |||
</td> | |||
<td> Simona Badau | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-20 | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-19 | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-20 | |||
</td></tr> | |||
<tr> | |||
<td><a href="https://wiki.mozilla.org/OS_X_10.7_support">OS X 10.7 support</a> - Not landed | |||
</td> | |||
<td> Vlad Ghetiu | |||
</td> | |||
<td> <b>N/A</b> - 2012-07-19 | |||
</td> | |||
<td> <b>N/A</b> - 2012-07-19 | |||
</td> | |||
<td> <b>N/A</b> - 2012-07-19 | |||
</td></tr> | |||
<tr> | |||
<td><a href="https://wiki.mozilla.org/Opt-in_activation_for_plugins">Opt-in activation for plugins</a> | |||
</td> | |||
<td> Paul Silaghi | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-20 | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-20 | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-20 | |||
</td></tr> | |||
<tr> | |||
<td><a href="https://wiki.mozilla.org/Speedy_Session_Restore">Speedy Session Restore</a> | |||
</td> | |||
<td> Paul Silaghi | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-19 | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-19 | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-19 | |||
</td></tr> | |||
<tr> | |||
<td><a href="https://wiki.mozilla.org/DevTools/Features/DeveloperToolbar">Developer Toolbar</a> | |||
</td> | |||
<td> Virgil Dicu | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-19 | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-19 | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-19 | |||
</td></tr> | |||
<tr> | |||
<td><a href="https://wiki.mozilla.org/DevTools/Features/GCLI">Command Line</a> | |||
</td> | |||
<td> Mihaela Velimiroviciu | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-20 | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-20 | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-20 | |||
</td></tr> | |||
<tr> | |||
<td><a href="https://wiki.mozilla.org/Show_PDF_inline">Show PDF Inline</a> | |||
</td> | |||
<td> Mihaela Velimiroviciu | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-20 | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-20 | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-20 | |||
</td></tr> | |||
<tr> | |||
<td><a href="Platform/Features/Incremental GC/TestPlan"> | |||
Incremental Garbage Collection</a> | |||
</td> | |||
<td> Ioana Budnar | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-19 | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-20 | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-20 | |||
</td></tr> | |||
</table> | |||
<p><b>Notes:</b> | |||
</p> | |||
<ul><li>OS X 10.7 support - not landed | |||
</li></ul> | |||
<h4>Smoke tests</h4> | |||
<p>Run the smoketests in MozTrap and report results below. | |||
</p> | |||
<table class="querytable" style="width:60%"> | |||
<tr> | |||
<td class="header"> Platform | |||
</td> | |||
<td class="header"> Assignee | |||
</td> | |||
<td class="header"> Result: <a href="https://moztrap.mozilla.org/results/cases/?filter-run=62">report</a> | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> Simona | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-19 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 64-bit | |||
</td> | |||
<td> Paul | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-20 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.7.4 | |||
</td> | |||
<td> VladG | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-19 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 32-bit | |||
</td> | |||
<td> Virgil | |||
</td> | |||
<td> <b>PASS</b> - 2012-07-19 | |||
</td></tr> | |||
</table> | |||
<h4>Known Issues</h4> | |||
<table class="bugzilla ui-helper-reset"> | |||
<thead> | |||
<tr> | |||
<th>ID</th> | |||
<th>Summary</th> | |||
<th>Status</th> | |||
<th>Priority</th> | |||
</tr> | |||
</thead> | |||
<tbody> | |||
<tr> | |||
<td><a href="https://bugzilla.mozilla.org/show_bug.cgi?id=692446">692446</a></td> | |||
<td><a href="https://bugzilla.mozilla.org/show_bug.cgi?id=692446">Firefox prompts to install windows media player plugin even though it is already installed</a></td> | |||
<td>NEW</td> | |||
<td>--</td> | |||
</tr> | |||
<tr> | |||
<td><a href="https://bugzilla.mozilla.org/show_bug.cgi?id=775491">775491</a></td> | |||
<td><a href="https://bugzilla.mozilla.org/show_bug.cgi?id=775491">HTML5 unresponsive script errors on YouTube (unresponsive script name: html5player-vflGWJljH.js)</a></td> | |||
<td>RESOLVED</td> | |||
<td>--</td> | |||
</tr> | |||
</tbody> | |||
</table> | |||
<p><br /> | |||
</p> | |||
<h3>Pre-beta Feature testing and Sign-off</h3> | |||
<p>Manual spot Checks [spreadsheet] | |||
</p> | |||
<ul><li>Platforms: Windows XP, Windows 7 x64, Ubuntu 11.10, Mac 10.6 | |||
</li><li>Found known issues: | |||
</li><li>New issues: | |||
</li><li>Status: | |||
</li></ul> | |||
<p>Run the <b>Firefox 16 Aurora Merge Feature Sign-Off</b> tests in <a href="https://moztrap.mozilla.org/runtests/">MozTrap</a> and report results below. | |||
</p> | |||
<table class="querytable"> | |||
<tr> | |||
<td class="header"> Feature | |||
</td> | |||
<td class="header"> Owner | |||
</td> | |||
<td class="header"> Win32 | |||
</td> | |||
<td class="header"> Mac | |||
</td> | |||
<td class="header"> Linux | |||
</td></tr> | |||
<tr> | |||
<td><a href="https://wiki.mozilla.org/In-content_preferences">In-content_preferences</a> | |||
</td> | |||
<td> Mihaela Velimiroviciu | |||
</td> | |||
<td> <b>disabled</b> - 2012-08-24 | |||
</td> | |||
<td> <b>disabled</b> - 2012-08-24 | |||
</td> | |||
<td> <b>disabled</b> - 2012-08-24 | |||
</td></tr> | |||
<tr> | |||
<td><a href="https://wiki.mozilla.org/User:P.A./Panel-based_Download_Manager">Panel Based Download Manager</a> | |||
</td> | |||
<td> Simona Badau | |||
</td> | |||
<td> <b>disabled</b> - 2012-08-24 | |||
</td> | |||
<td> <b>disabled</b> - 2012-08-24 | |||
</td> | |||
<td> <b>disabled</b> - 2012-08-24 | |||
</td></tr> | |||
<tr> | |||
<td><a href="https://wiki.mozilla.org/OS_X_10.7_support">OS X 10.7 support</a> | |||
</td> | |||
<td> Vlad Ghetiu | |||
</td> | |||
<td> <b>NA</b> - 2012-08-24 | |||
</td> | |||
<td> <b>NA</b> - 2012-08-24 | |||
</td> | |||
<td> <b>NA</b> - 2012-08-24 | |||
</td></tr> | |||
<tr> | |||
<td><a href="https://wiki.mozilla.org/Opt-in_activation_for_plugins">Opt-in activation for plugins</a> | |||
</td> | |||
<td> Paul Silaghi | |||
</td> | |||
<td> <b>disabled</b> - 2012-08-24 | |||
</td> | |||
<td> <b>disabled</b> - 2012-08-24 | |||
</td> | |||
<td> <b>disabled</b> - 2012-08-24 | |||
</td></tr> | |||
<tr> | |||
<td><a href="https://wiki.mozilla.org/Speedy_Session_Restore">Speedy Session Restore</a> | |||
</td> | |||
<td> Paul Silaghi | |||
</td> | |||
<td> <b>NA</b> - 2012-08-24 | |||
</td> | |||
<td> <b>NA</b> - 2012-08-24 | |||
</td> | |||
<td> <b>NA</b> - 2012-08-24 | |||
</td></tr> | |||
<tr> | |||
<td><a href="https://wiki.mozilla.org/DevTools/Features/DeveloperToolbar">Developer Toolbar</a> | |||
</td> | |||
<td> Virgil Dicu | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-24 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-24 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-24 | |||
</td></tr> | |||
<tr> | |||
<td><a href="https://wiki.mozilla.org/DevTools/Features/GCLI">Command Line</a> | |||
</td> | |||
<td> Mihaela Velimiroviciu | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-24 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-24 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-24 | |||
</td></tr> | |||
<tr> | |||
<td><a href="https://wiki.mozilla.org/Show_PDF_inline">Show PDF Inline</a> | |||
</td> | |||
<td> Mihaela Velimiroviciu | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-24 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-24 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-24 | |||
</td></tr> | |||
<tr> | |||
<td><a href="Platform/Features/Incremental GC/TestPlan"> | |||
Incremental Garbage Collection</a> | |||
</td> | |||
<td> Ioana Budnar | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-24 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-24 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-24 | |||
</td></tr> | |||
</table> | |||
<p><b>Notes:</b> | |||
</p> | |||
<ul><li> OS X 10.7 support: Not yet implemented. | |||
</li><li> Speedy Session Restore: backed out. | |||
</li></ul> | |||
<h4>Smoke tests</h4> | |||
<p>Run the smoketests in MozTrap and report results below. | |||
</p> | |||
<table class="querytable" style="width:60%"> | |||
<tr> | |||
<td class="header"> Platform | |||
</td> | |||
<td class="header"> Assignee | |||
</td> | |||
<td class="header"> Result: <a href="https://moztrap.mozilla.org/results/cases/?filter-run=62">report</a> | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> Simona | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-24 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 64-bit | |||
</td> | |||
<td> VladG | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-24 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.7.4 | |||
</td> | |||
<td> VladG | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-24 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 32-bit | |||
</td> | |||
<td> Virgil | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-24 | |||
</td></tr> | |||
</table> | |||
<h4>Known Issues</h4> | |||
<table class="bugzilla ui-helper-reset"> | |||
<thead> | |||
<tr> | |||
<th>ID</th> | |||
<th>Summary</th> | |||
<th>Status</th> | |||
<th>Priority</th> | |||
</tr> | |||
</thead> | |||
<tbody> | |||
<tr> | |||
<td><a href="https://bugzilla.mozilla.org/show_bug.cgi?id=775082">775082</a></td> | |||
<td><a href="https://bugzilla.mozilla.org/show_bug.cgi?id=775082">Video controls disappear after dragging the tabs</a></td> | |||
<td>NEW</td> | |||
<td>--</td> | |||
</tr> | |||
<tr> | |||
<td><a href="https://bugzilla.mozilla.org/show_bug.cgi?id=784289">784289</a></td> | |||
<td><a href="https://bugzilla.mozilla.org/show_bug.cgi?id=784289">Inspector button remains open in devtoolbar when closing the Inspector via x button</a></td> | |||
<td>RESOLVED</td> | |||
<td>--</td> | |||
</tr> | |||
</tbody> | |||
</table> | |||
<p><br /> | |||
</p> | |||
<h2> Beta 1 </h2> | |||
<table class="querytable" style="width:30%"> | |||
<tr> | |||
<td style="width:10%"> | |||
</td> | |||
<td style="width:20%"> | |||
</td></tr> | |||
<tr> | |||
<td> <b>Builds</b> | |||
</td> | |||
<td> <a href="ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b1-candidates/build1/">FTP</a>, <a href="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b1-candidates/build1/">HTTP</a> | |||
</td></tr> | |||
<tr> | |||
<td> <b>ID</b> | |||
</td> | |||
<td> 20120828083259 | |||
</td></tr> | |||
<tr> | |||
<td> <b>Changeset</b> | |||
</td> | |||
<td> <a href="http://hg.mozilla.org/releases/mozilla-beta/rev/c4e1ca772267">c4e1ca772267</a> (<a href="http://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=2099c2bea208&tochange=c4e1ca772267">pushlog</a>) | |||
</td></tr></table> | |||
<dl><dt> Checklist | |||
</dt></dl> | |||
<ul><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Automated_Testing">Automated Testing</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Manual_Testing">Manual Testing</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Update_Testing">Update Testing</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Bug_Triage">Bug Triage</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> Peer Sign-off | |||
</li><li> <span class="fck_mw_template">{{ |}}</span> Final sign-off sent and updates qualified on Beta channel | |||
</li></ul> | |||
<h3> Automated Testing </h3> | |||
<p>The following are the detailed results of the automated Mozmill test runs. | |||
</p> | |||
<table class="querytable" style="width:50%"> | |||
== | <tr> | ||
<td class="header" style="width:20%"> Platform | |||
</td> | |||
<td class="header" style="width:30%"> PASS: <a href="http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-08-28&to=2012-08-28">report</a> | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-28 | |||
</td></tr> | |||
<tr> | |||
<td> Windows Vista | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-28 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-28 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-28 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 32-bit | |||
</td> | |||
<td> <b>N/A</b> (pending IT) | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 64-bit | |||
</td> | |||
<td> <b>N/A</b> (pending IT) | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.6 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-28 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.7 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-28 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.8 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-28 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-28 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-28 | |||
</td></tr></table> | |||
<dl><dt> Test Failures (<a href="https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA">known bugs</a>) | |||
</dt></dl> | |||
<h3> Manual Testing </h3> | |||
<table class="querytable"> | |||
<tr> | |||
<td class="header"> Operating System | |||
</td> | |||
<td class="header"> Tester | |||
</td> | |||
<td class="header"> Plug-in Compatibility | |||
</td> | |||
<td class="header"> Add-on Compatibility | |||
</td> | |||
<td class="header"> Web Compatibility | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> Simona | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 | |||
</td> | |||
<td> Mihaela | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 | |||
</td> | |||
<td> Paul | |||
</td> | |||
<td><b>PASS</b> - 2012-08-29 | |||
</td> | |||
<td><b>PASS</b> - 2012-08-29 | |||
</td> | |||
<td><b>PASS</b> - 2012-08-29 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X | |||
</td> | |||
<td> VladG | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 | |||
</td> | |||
<td> Virgil | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td></tr> | |||
<tr> | |||
<td class="header"> Operating System | |||
</td> | |||
<td class="header"> Tester | |||
</td> | |||
<td class="header"> Web Mail | |||
</td> | |||
<td class="header"> Mac 10.8 Signing | |||
</td> | |||
<td class="header"> Breakpad | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> Simona | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 | |||
</td> | |||
<td> Mihaela | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 | |||
</td> | |||
<td> Paul | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X | |||
</td> | |||
<td> VladG | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 | |||
</td> | |||
<td> Virgil | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td></tr></table> | |||
<h4> Plug-in Compatibility </h4> | |||
<p>Install the following plug-ins and go to websites which use them, looking for rendering issues, functional issues, performance issues, and crashes. | |||
</p> | |||
<ul><li> Adobe Flash Player | |||
<ul><li><a href="http://get.adobe.com/flashplayer/">11.3 Release</a> | |||
</li><li><a href="http://labs.adobe.com/downloads/flashplayer11-4.html">Flash 11.4 beta</a> (make sure you can autoupdate) | |||
</li></ul> | |||
</li><li> <a href="http://www.microsoft.com/silverlight/">Microsoft Silverlight</a> | |||
</li><li> <a href="http://www.real.com/">RealPlayer</a> | |||
<ul><li> Real Player plugin crashed when opening pages with content managed by the plugin: <a href="http://www.slp3d2.com/launchsiteglobal/test.cfm">click</a>, <a href="http://cf.unc.edu/skillstest/take/realplayer.htm">click</a> - <span class="fck_mw_template">{{bug|667399}}</span> | |||
</li></ul> | |||
</li><li> <a href="http://www.apple.com/quicktime/download/">QuickTime</a> | |||
</li></ul> | |||
<h4> Add-on Compatibility </h4> | |||
<p>Test the following websites for rendering issues, functional issues, performance issues, and crashes. Pay particular attention to any problems seen after updating Firefox with add-ons installed. | |||
</p> | |||
<ul><li> <a href="https://addons.mozilla.org/en-US/firefox/compatibility/15.0">Top AMO add-ons</a> | |||
</li><li> AVG toolbar (installed via AVG software) | |||
<ul><li> AVG Safe Search 12.0.0.1912 is not compatible with Firefox 15 beta. | |||
</li><li> AVG Security Toolbar can't be installed on Win XP - we get a blue screen while installing it. | |||
</li></ul> | |||
</li><li> Norton toolbar (installed via Norton software) | |||
<ul><li> Not currently available. Investigating if we can get this software. | |||
</li></ul> | |||
</li><li> <a href="http://sg.toolbar.yahoo.com/">Yahoo toolbar</a> | |||
</li><li> <a href="http://bar.yandex.ru/en/">Yandex toolbar</a> | |||
</li><li> <a href="http://babylon-toolbar.software.informer.com/download/">Babylon toolbar</a> | |||
</li><li> <a href="http://lavasoft.com/">Ad-Aware Antivirus</a> | |||
<ul><li> The toolbar can't be installed on Firefox. | |||
</li></ul> | |||
</li></ul> | |||
<h4> Web Compatibility </h4> | |||
<p>Test the following websites for rendering issues, functional issues, performance issues, and crashes | |||
</p> | |||
<ol><li> <a href="http://www.alexa.com/topsites">Overall Top 10</a> | |||
</li><li> <a href="http://www.alexa.com/topsites/countries">Top by Country</a> (pick a random site from three random countries) | |||
</li><li> <a href="http://www.alexa.com/topsites/category">Top by Category</a> (pick a random site from three random categories) | |||
</li><li> Webmail clients like gmail, hotmail, and yahoo mail (pay particular attention to any built in web messenger software) | |||
</li><li> Try Google's <a href="http://support.google.com/maps/bin/answer.py?hl=en&answer=1630790">MapsGL demo</a> | |||
</li><li> Try YouTube's <a href="http://www.youtube.com/html5">HTML5 demo</a> | |||
</li></ol> | |||
<h4> Web Mail </h4> | |||
<p>Test the following websites for rendering issues, functional issues, performance issues, and crashes. In particular make sure email, attachments, web messenger, and video chat are working as expected. | |||
</p> | |||
<ol><li> <a href="https://mail.google.com/">Google Mail</a> | |||
</li><li> <a href="https://mail.yahoo.com/">Yahoo Mail</a> | |||
</li><li> <a href="https://www.hotmail.com">Hotmail</a> | |||
</li><li> <a href="https://www.outlook.com/">Outlook.com</a> | |||
</li></ol> | |||
<h4> Mac 10.8 Signing </h4> | |||
<p>NOTE: Test needs to be run from a clean Mac OSX 10.8 install. Please keep a vanilla image handy so you can restore once completing this test (<a href="https://people.mozilla.com/~stmichaud/Gatekeeper/MLGatekeeper.html">more info</a>). | |||
</p> | |||
<ul><li> Download the .dmg file and drag the Firefox.app file to a location on your computer | |||
</li><li> Open a terminal window and cd to the folder Firefox.app is located | |||
</li><li> Run <i>codesign -vv Firefox.app</i> in terminal, output should contain <i>valid on disk</i> and <i>satisfies its Designated Requirement</i> | |||
</li><li> Run <i>xattr Firefox.app</i> in terminal, output should contain <i>com.apple.quarantine</i> | |||
</li><li> Double click the Firefox.app icon, you should see a warning about the program being provided by an <i>unknown developer</i> or having been <i>downloaded from the Internet</i> | |||
</li><li> Click "open" and browse to any website, Firefox should start and you should be able to browse | |||
</li><li> Quit Firefox and double click the Firefox.app icon again, you should <i>not</i> be reprompted for the warning in step 5, firefox should start up and browse without any issues | |||
</li></ul> | |||
<p>Spot check build after this test | |||
</p> | |||
<ul><li> Try all the menu item functions | |||
</li><li> Gestures | |||
</li><li> Plug-in compatibility | |||
</li><li> Top-site compatibility | |||
</li><li> Add-on compatibility | |||
</li><li> Software update | |||
</li></ul> | |||
<h4> Breakpad End-to-End </h4> | |||
<p>Test the end-to-end functionality in the crash reporting feature, across operating systems. | |||
</p> | |||
<ul><li> Use the <a href="http://ted.mielczarek.org/mozilla/crashme.html">Crash Me add-on</a> to trigger a crash | |||
</li><li> Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc | |||
</li><li> Repeatedly crash Firefox and submit your reports | |||
</li><li> Verify that your session restores properly | |||
</li><li> Verify your crash reports are submitted properly by going through about:crashes and clicking the reports | |||
</li><li> Try adding comments to your crash report and verify the crash-stats.com report shows the comment | |||
</li></ul> | |||
<h3> Update Testing </h3> | |||
<ul><li> Partial: 15.0b6 | |||
</li><li> Complete: 15.0b5 and earlier | |||
</li></ul> | |||
<table class="querytable" style="width:60%"> | |||
== | <tr> | ||
=== | <td class="header"> Platform | ||
; | </td> | ||
<td class="header"> Releasetest: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-08-28&to=2012-08-30&target=16.0">report</a> | |||
</td> | |||
<td class="header"> Beta: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-08-28&to=2012-08-30&target=16.0">report</a> | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-30 | |||
</td></tr> | |||
<tr> | |||
<td> Windows Vista | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-30 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-30 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-30 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 32-bit | |||
</td> | |||
<td> <b>manual</b> - see below | |||
</td> | |||
<td> <b>manual</b> - see below | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 64-bit | |||
</td> | |||
<td> <b>manual</b> - see below | |||
</td> | |||
<td> <b>manual</b> - see below | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.6 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-30 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.7 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-30 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.8 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-30 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-30 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-29 | |||
</td> | |||
<td> <b>PASS</b> - 2012-08-30 | |||
</td></tr></table> | |||
<p><i>Windows 8: manual spotcheck until automation in place</i> | |||
</p> | |||
<ul><li> RELEASETEST | |||
<ul><li> <b>PASS</b> - Firefox 15.0b6 en-US (partial) | |||
</li><li> <b>PASS</b> - Firefox 15.0b6 de (partial) | |||
</li><li> <b>PASS</b> - Firefox 14.0b6 en-US (complete) | |||
</li><li> <b>PASS</b> - Firefox 14.0b6 zh-TW (complete) | |||
</li></ul> | |||
</li><li> BETA | |||
<ul><li> <b>PASS</b> - Firefox 15.0b6 en-US (partial) | |||
</li><li> <b>PASS</b> - Firefox 15.0b6 de (partial) | |||
</li><li> <b>PASS</b> - Firefox 14.0b6 en-US (complete) | |||
</li><li> <b>PASS</b> - Firefox 14.0b6 zh-TW (complete) | |||
</li></ul> | |||
</li></ul> | |||
<dl><dt> Issues | |||
</dt></dl> | |||
<h3> Bug Triage </h3> | |||
<ol><li> <a href="http://bit.ly/RquQy9">VERIFYME</a> (3): verify the fix and do some exploratory testing around the issue | |||
</li><li> <a href="http://bit.ly/Ppe9E0">FIXED</a> (221): triage these bugs and test to ensure the fix is effective | |||
</li><li> <a href="http://bit.ly/PMaYVx">QAWANTED</a> (3): triage these bugs and test to fulfill the QA request | |||
</li><li> <a href="http://bit.ly/OtEqj1">UNCONFIRMED</a> (3): triage these bugs for potential new regressions | |||
</li></ol> | |||
<dl><dt> Results | |||
</dt></dl> | |||
<ul><li> <a href="http://bit.ly/MXYdqS">VERIFIED</a>: 50/221 | |||
</li><li> <a href="http://bit.ly/POyxuA">WONTVERIFY</a>: 21/221 | |||
</li><li> <a href="http://bit.ly/O09WSK">HELPWANTED</a>: 14/221 | |||
</li></ul> | |||
<h2> Beta 2 </h2> | |||
<table class="querytable" style="width:30%"> | |||
<tr> | |||
<td style="width:10%"> | |||
</td> | |||
<td style="width:20%"> | |||
</td></tr> | |||
<tr> | |||
<td> <b>Builds</b> | |||
</td> | |||
<td> <a href="ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b2-candidates/build1/">FTP</a>, <a href="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b2-candidates/build1/">HTTP</a> | |||
</td></tr> | |||
<tr> | |||
<td> <b>ID</b> | |||
</td> | |||
<td> 20120904124322 | |||
</td></tr> | |||
<tr> | |||
<td> <b>Changeset</b> | |||
</td> | |||
<td> <a href="http://hg.mozilla.org/releases/mozilla-beta/rev/cad589192f3c">cad589192f3c</a> (<a href="http://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=c4e1ca772267&tochange=cad589192f3c">pushlog</a>) | |||
</td></tr></table> | |||
<dl><dt> Checklist | |||
</dt></dl> | |||
<ul><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Automated_Testing_2">Automated Testing</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Manual_Testing_2">Manual Testing</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Update_Testing_2">Update Testing</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Bug_Triage_2">Bug Triage</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> Peer Sign-off | |||
</li><li> <span class="fck_mw_template">{{ |}}</span> Final sign-off sent and updates qualified on Beta channel | |||
</li></ul> | |||
<h3> Automated Testing 2 </h3> | |||
<p>The following are the detailed results of the automated Mozmill test runs. | |||
</p> | |||
<table class="querytable" style="width:50%"> | |||
; Update | <tr> | ||
<td class="header" style="width:20%"> Platform | |||
</td> | |||
<td class="header" style="width:30%"> PASS: <a href="http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-09-04&to=2012-09-05">report</a> | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-05 | |||
</td></tr> | |||
<tr> | |||
<td> Windows Vista | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-05 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-05 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-05 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 32-bit | |||
</td> | |||
<td> <b>N/A</b> (pending IT) | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 64-bit | |||
</td> | |||
<td> <b>N/A</b> (pending IT) | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.6 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-05 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.7 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-05 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.8 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-05 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-05 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-05 | |||
</td></tr></table> | |||
<dl><dt> Test Failures (<a href="https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA">known bugs</a>) | |||
</dt></dl> | |||
<h3> Manual Testing 2 </h3> | |||
<table class="querytable"> | |||
<tr> | |||
<td class="header"> Operating System | |||
</td> | |||
<td class="header"> Tester | |||
</td> | |||
<td class="header"> Plug-in Compatibility | |||
</td> | |||
<td class="header"> Add-on Compatibility | |||
</td> | |||
<td class="header"> Web Compatibility | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> Ioana | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 | |||
</td> | |||
<td> Mihaela | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 | |||
</td> | |||
<td> Virgil | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X | |||
</td> | |||
<td> VladG | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 | |||
</td> | |||
<td> Ioana | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td></tr> | |||
<tr> | |||
<td class="header"> Operating System | |||
</td> | |||
<td class="header"> Tester | |||
</td> | |||
<td class="header"> Web Mail | |||
</td> | |||
<td class="header"> Mac 10.8 Signing | |||
</td> | |||
<td class="header"> Breakpad | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> Virgil | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 | |||
</td> | |||
<td> Mihaela | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 | |||
</td> | |||
<td> Virgil | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X | |||
</td> | |||
<td> VladG | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 | |||
</td> | |||
<td> Ioana | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-07 | |||
</td></tr></table> | |||
<h4> Plug-in Compatibility 2 </h4> | |||
<p>Install the following plug-ins and go to websites which use them, looking for rendering issues, functional issues, performance issues, and crashes. | |||
</p> | |||
<ul><li> Adobe Flash Player | |||
<ul><li><a href="http://get.adobe.com/flashplayer/">11.4 Release</a> | |||
</li></ul> | |||
</li><li> <a href="http://www.microsoft.com/silverlight/">Microsoft Silverlight</a> | |||
</li><li> <a href="http://www.real.com/">RealPlayer</a> | |||
<ul><li> Real Player plugin crashed when opening pages with content managed by the plugin: <a href="http://www.slp3d2.com/launchsiteglobal/test.cfm">click</a>, <a href="http://cf.unc.edu/skillstest/take/realplayer.htm">click</a> - <span class="fck_mw_template">{{bug|667399}}</span> | |||
</li></ul> | |||
</li><li> <a href="http://www.apple.com/quicktime/download/">QuickTime</a> | |||
</li></ul> | |||
<h4> Add-on Compatibility 2 </h4> | |||
<p>Test the following websites for rendering issues, functional issues, performance issues, and crashes. Pay particular attention to any problems seen after updating Firefox with add-ons installed. | |||
</p> | |||
<ul><li> <a href="https://addons.mozilla.org/en-US/firefox/compatibility/15.0">Top AMO add-ons</a> | |||
</li><li> AVG toolbar (installed via AVG software) | |||
</li><li> Norton toolbar (installed via Norton software) | |||
</li><li> <a href="http://sg.toolbar.yahoo.com/">Yahoo toolbar</a> | |||
</li><li> <a href="http://bar.yandex.ru/en/">Yandex toolbar</a> | |||
</li><li> <a href="http://babylon-toolbar.software.informer.com/download/">Babylon toolbar</a> | |||
</li><li> <a href="http://lavasoft.com/">Ad-Aware Antivirus</a> | |||
</li></ul> | |||
<h4> Web Compatibility 2 </h4> | |||
<p>Test the following websites for rendering issues, functional issues, performance issues, and crashes | |||
</p> | |||
<ol><li> <a href="http://www.alexa.com/topsites">Overall Top 10</a> | |||
</li><li> <a href="http://www.alexa.com/topsites/countries">Top by Country</a> (pick a random site from three random countries) | |||
</li><li> <a href="http://www.alexa.com/topsites/category">Top by Category</a> (pick a random site from three random categories) | |||
</li><li> Webmail clients like gmail, hotmail, and yahoo mail (pay particular attention to any built in web messenger software) | |||
</li><li> Try Google's <a href="http://support.google.com/maps/bin/answer.py?hl=en&answer=1630790">MapsGL demo</a> | |||
</li><li> Try YouTube's <a href="http://www.youtube.com/html5">HTML5 demo</a> | |||
</li></ol> | |||
<h4> Web Mail 2 </h4> | |||
<p>Test the following websites for rendering issues, functional issues, performance issues, and crashes. In particular make sure email, attachments, web messenger, and video chat are working as expected. | |||
</p> | |||
<ol><li> <a href="https://mail.google.com/">Google Mail</a> | |||
</li><li> <a href="https://mail.yahoo.com/">Yahoo Mail</a> | |||
</li><li> <a href="https://www.hotmail.com">Hotmail</a> | |||
</li><li> <a href="https://www.outlook.com/">Outlook.com</a> | |||
</li></ol> | |||
<h4> Mac 10.8 Signing 2 </h4> | |||
<p>NOTE: Test needs to be run from a clean Mac OSX 10.8 install. Please keep a vanilla image handy so you can restore once completing this test (<a href="https://people.mozilla.com/~stmichaud/Gatekeeper/MLGatekeeper.html">more info</a>). | |||
</p> | |||
<ul><li> Download the .dmg file and drag the Firefox.app file to a location on your computer | |||
</li><li> Open a terminal window and cd to the folder Firefox.app is located | |||
</li><li> Run <i>codesign -vv Firefox.app</i> in terminal, output should contain <i>valid on disk</i> and <i>satisfies its Designated Requirement</i> | |||
</li><li> Run <i>xattr Firefox.app</i> in terminal, output should contain <i>com.apple.quarantine</i> | |||
</li><li> Double click the Firefox.app icon, you should see a warning about the program being provided by an <i>unknown developer</i> or having been <i>downloaded from the Internet</i> | |||
</li><li> Click "open" and browse to any website, Firefox should start and you should be able to browse | |||
</li><li> Quit Firefox and double click the Firefox.app icon again, you should <i>not</i> be reprompted for the warning in step 5, firefox should start up and browse without any issues | |||
</li></ul> | |||
<p>Spot check build after this test | |||
</p> | |||
<ul><li> Try all the menu item functions | |||
</li><li> Gestures | |||
</li><li> Plug-in compatibility | |||
</li><li> Top-site compatibility | |||
</li><li> Add-on compatibility | |||
</li><li> Software update | |||
</li></ul> | |||
<h4> Breakpad End-to-End 2 </h4> | |||
<p>Test the end-to-end functionality in the crash reporting feature, across operating systems. | |||
</p> | |||
<ul><li> Use the <a href="http://ted.mielczarek.org/mozilla/crashme.html">Crash Me add-on</a> to trigger a crash | |||
</li><li> Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc | |||
</li><li> Repeatedly crash Firefox and submit your reports | |||
</li><li> Verify that your session restores properly | |||
</li><li> Verify your crash reports are submitted properly by going through about:crashes and clicking the reports | |||
</li><li> Try adding comments to your crash report and verify the crash-stats.com report shows the comment | |||
</li></ul> | |||
<h3> Update Testing 2 </h3> | |||
<ul><li> Partial: 16.0b1 | |||
</li><li> Complete: 15.0b6 and earlier | |||
</li></ul> | |||
<table class="querytable" style="width:60%"> | |||
=== | <tr> | ||
' | <td class="header"> Platform | ||
</td> | |||
<td class="header"> Releasetest: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-09-05&to=2012-09-06&target=16.0">report</a> | |||
</td> | |||
<td class="header"> Beta: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-09-06&to=2012-09-07&target=16.0">report</a> | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-06 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-06 | |||
</td></tr> | |||
<tr> | |||
<td> Windows Vista | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-06 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-06 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-06 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-06 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-06 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-06 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 32-bit | |||
</td> | |||
<td> <b>manual</b> - see below | |||
</td> | |||
<td> <b>manual</b> - see below | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 64-bit | |||
</td> | |||
<td> <b>PASS</b> - see below | |||
</td> | |||
<td> <b>PASS</b> - see below | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.6 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-06 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-06 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.7 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-06 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-06 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.8 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-06 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-06 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-06 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-06 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-06 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-06 | |||
</td></tr></table> | |||
<p><i>Windows 8: manual spotcheck until automation in place</i> | |||
</p> | |||
<ul><li> RELEASETEST | |||
<ul><li> <b>PASS</b> - Firefox 16.0b1 en-US (partial) | |||
</li><li> <b>PASS</b> - Firefox 15.0b6 de (complete) | |||
</li></ul> | |||
</li><li> BETA | |||
<ul><li> <b>PASS</b> - Firefox 16.0b1 en-US (partial) | |||
</li><li> <b>PASS</b> - Firefox 15.0b6 de (complete) | |||
</li></ul> | |||
</li></ul> | |||
<dl><dt> Issues | |||
</dt></dl> | |||
<h3> Bug Triage 2 </h3> | |||
<ol><li> <a href="http://bit.ly/RquQy9">VERIFYME</a> : verify the fix and do some exploratory testing around the issue | |||
</li><li> <a href="http://bit.ly/Ppe9E0">FIXED</a> : triage these bugs and test to ensure the fix is effective | |||
</li><li> <a href="http://bit.ly/PMaYVx">QAWANTED</a> : triage these bugs and test to fulfill the QA request | |||
</li><li> <a href="http://bit.ly/OtEqj1">UNCONFIRMED</a> : triage these bugs for potential new regressions | |||
</li></ol> | |||
<dl><dt> Results | |||
</dt></dl> | |||
<ul><li> <a href="http://bit.ly/MXYdqS">VERIFIED</a>: Verified/Fixed Ratio | |||
</li><li> <a href="http://bit.ly/POyxuA">WONTVERIFY</a>: Won't Verify/Fixed Ratio | |||
</li><li> <a href="http://bit.ly/O09WSK">HELPWANTED</a>: Help Wanted/Fixed Ratio | |||
</li></ul> | |||
<h2> Beta 3 </h2> | |||
<table class="querytable" style="width:30%"> | |||
<tr> | |||
<td style="width:10%"> | |||
</td> | |||
<td style="width:20%"> | |||
</td></tr> | |||
<tr> | |||
<td> <b>Builds</b> | |||
</td> | |||
<td> <a href="ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b3-candidates/build1/">FTP</a>, <a href="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b3-candidates/build1/">HTTP</a> | |||
</td></tr> | |||
<tr> | |||
<td> <b>ID</b> | |||
</td> | |||
<td> 20120911232325 | |||
</td></tr> | |||
<tr> | |||
<td> <b>Changeset</b> | |||
</td> | |||
<td> <a href="http://hg.mozilla.org/releases/mozilla-beta/rev/de3f06a549de">de3f06a549de</a> (<a href="http://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=cad589192f3c&tochange=de3f06a549de">pushlog</a>) | |||
</td></tr></table> | |||
<dl><dt> Checklist | |||
</dt></dl> | |||
<ul><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Automated_Testing_2">Automated Testing</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Manual_Testing_2">Manual Testing</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Update_Testing_2">Update Testing</a> | |||
</li><li> <span class="fck_mw_template">{{ok|}}</span> <a href="Releases/Firefox 16/Test Plan#Bug_Triage_2">Bug Triage</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> Peer Sign-off | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> Final sign-off sent and updates qualified on Beta channel | |||
</li></ul> | |||
<h3> Automated Testing 3 </h3> | |||
<p>The following are the detailed results of the automated Mozmill test runs. | |||
</p> | |||
<table class="querytable" style="width:50%"> | |||
<tr> | |||
<td class="header" style="width:20%"> Platform | |||
</td> | |||
<td class="header" style="width:30%"> PASS: <a href="http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-09-11&to=2012-09-12">report</a> | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td></tr> | |||
<tr> | |||
<td> Windows Vista | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.6 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.7 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.8 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td></tr></table> | |||
<dl><dt> Test Failures (<a href="https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA">known bugs</a>) | |||
</dt></dl> | |||
<h3> Manual Testing 3 </h3> | |||
<table class="querytable"> | |||
<tr> | |||
<td class="header"> Operating System | |||
</td> | |||
<td class="header"> Tester | |||
</td> | |||
<td class="header"> Plug-in Compatibility | |||
</td> | |||
<td class="header"> Add-on Compatibility | |||
</td> | |||
<td class="header"> Web Compatibility | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> Mihaela | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 | |||
</td> | |||
<td> Ioana | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 | |||
</td> | |||
<td> Paul | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X | |||
</td> | |||
<td> VladG | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 | |||
</td> | |||
<td> Virgil | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td></tr> | |||
<tr> | |||
<td class="header"> Operating System | |||
</td> | |||
<td class="header"> Tester | |||
</td> | |||
<td class="header"> Web Mail | |||
</td> | |||
<td class="header"> Mac 10.8 Signing | |||
</td> | |||
<td class="header"> Breakpad | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> Mihaela | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 | |||
</td> | |||
<td> Ioana | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 | |||
</td> | |||
<td> Paul | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X | |||
</td> | |||
<td> VladG | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 | |||
</td> | |||
<td> Virgil | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-12 | |||
</td></tr></table> | |||
<h4> Plug-in Compatibility 3 </h4> | |||
<p>Install the following plug-ins and go to websites which use them, looking for rendering issues, functional issues, performance issues, and crashes. | |||
</p> | |||
<ul><li> Adobe Flash Player | |||
<ul><li><a href="http://get.adobe.com/flashplayer/">11.4 Release</a> | |||
</li></ul> | |||
</li><li> <a href="http://www.microsoft.com/silverlight/">Microsoft Silverlight</a> | |||
</li><li> <a href="http://www.real.com/">RealPlayer</a> | |||
<ul><li> Real Player plugin crashed when opening pages with content managed by the plugin: <a href="http://www.slp3d2.com/launchsiteglobal/test.cfm">click</a>, <a href="http://cf.unc.edu/skillstest/take/realplayer.htm">click</a> - <span class="fck_mw_template">{{bug|667399}}</span> | |||
</li></ul> | |||
</li><li> <a href="http://www.apple.com/quicktime/download/">QuickTime</a> | |||
</li></ul> | |||
<h4> Add-on Compatibility 3 </h4> | |||
<p>Test the following websites for rendering issues, functional issues, performance issues, and crashes. Pay particular attention to any problems seen after updating Firefox with add-ons installed. | |||
</p> | |||
<ul><li> <a href="https://addons.mozilla.org/en-US/firefox/compatibility/15.0">Top AMO add-ons</a> | |||
</li><li> AVG toolbar (installed via AVG software) | |||
</li><li> Norton toolbar (installed via Norton software) | |||
</li><li> <a href="http://sg.toolbar.yahoo.com/">Yahoo toolbar</a> | |||
</li><li> <a href="http://bar.yandex.ru/en/">Yandex toolbar</a> | |||
</li><li> <a href="http://babylon-toolbar.software.informer.com/download/">Babylon toolbar</a> | |||
</li><li> <a href="http://lavasoft.com/">Ad-Aware Antivirus</a> | |||
</li></ul> | |||
<h4> Web Compatibility 3 </h4> | |||
<p>Test the following websites for rendering issues, functional issues, performance issues, and crashes | |||
</p> | |||
<ol><li> <a href="http://www.alexa.com/topsites">Overall Top 10</a> | |||
</li><li> <a href="http://www.alexa.com/topsites/countries">Top by Country</a> (pick a random site from three random countries) | |||
</li><li> <a href="http://www.alexa.com/topsites/category">Top by Category</a> (pick a random site from three random categories) | |||
</li><li> Webmail clients like gmail, hotmail, and yahoo mail (pay particular attention to any built in web messenger software) | |||
</li><li> Try Google's <a href="http://support.google.com/maps/bin/answer.py?hl=en&answer=1630790">MapsGL demo</a> | |||
</li><li> Try YouTube's <a href="http://www.youtube.com/html5">HTML5 demo</a> | |||
</li></ol> | |||
<h4> Web Mail 3 </h4> | |||
<p>Test the following websites for rendering issues, functional issues, performance issues, and crashes. In particular make sure email, attachments, web messenger, and video chat are working as expected. | |||
</p> | |||
<ol><li> <a href="https://mail.google.com/">Google Mail</a> | |||
</li><li> <a href="https://mail.yahoo.com/">Yahoo Mail</a> | |||
</li><li> <a href="https://www.hotmail.com">Hotmail</a> | |||
</li><li> <a href="https://www.outlook.com/">Outlook.com</a> | |||
</li></ol> | |||
<h4> Mac 10.8 Signing 3 </h4> | |||
<p>NOTE: Test needs to be run from a clean Mac OSX 10.8 install. Please keep a vanilla image handy so you can restore once completing this test (<a href="https://people.mozilla.com/~stmichaud/Gatekeeper/MLGatekeeper.html">more info</a>). | |||
</p> | |||
<ul><li> Download the .dmg file and drag the Firefox.app file to a location on your computer | |||
</li><li> Open a terminal window and cd to the folder Firefox.app is located | |||
</li><li> Run <i>codesign -vv Firefox.app</i> in terminal, output should contain <i>valid on disk</i> and <i>satisfies its Designated Requirement</i> | |||
</li><li> Run <i>xattr Firefox.app</i> in terminal, output should contain <i>com.apple.quarantine</i> | |||
</li><li> Double click the Firefox.app icon, you should see a warning about the program being provided by an <i>unknown developer</i> or having been <i>downloaded from the Internet</i> | |||
</li><li> Click "open" and browse to any website, Firefox should start and you should be able to browse | |||
</li><li> Quit Firefox and double click the Firefox.app icon again, you should <i>not</i> be reprompted for the warning in step 5, firefox should start up and browse without any issues | |||
</li></ul> | |||
<p>Spot check build after this test | |||
</p> | |||
<ul><li> Try all the menu item functions | |||
</li><li> Gestures | |||
</li><li> Plug-in compatibility | |||
</li><li> Top-site compatibility | |||
</li><li> Add-on compatibility | |||
</li><li> Software update | |||
</li></ul> | |||
<h4> Breakpad End-to-End 3 </h4> | |||
<p>Test the end-to-end functionality in the crash reporting feature, across operating systems. | |||
</p> | |||
<ul><li> Use the <a href="http://ted.mielczarek.org/mozilla/crashme.html">Crash Me add-on</a> to trigger a crash | |||
</li><li> Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc | |||
</li><li> Repeatedly crash Firefox and submit your reports | |||
</li><li> Verify that your session restores properly | |||
</li><li> Verify your crash reports are submitted properly by going through about:crashes and clicking the reports | |||
</li><li> Try adding comments to your crash report and verify the crash-stats.com report shows the comment | |||
</li></ul> | |||
<h3> Update Testing 3 </h3> | |||
<ul><li> Partial: 16.0b2 | |||
</li><li> Complete: 15.0b6 and earlier | |||
</li></ul> | |||
<table class="querytable" style="width:60%"> | |||
=== | <tr> | ||
<td class="header"> Platform | |||
</td> | |||
<td class="header"> Releasetest: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-09-11&to=2012-09-13&target=16.0">report</a> | |||
</td> | |||
<td class="header"> Beta: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-09-11&to=2012-09-13&target=16.0">report</a> | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td></tr> | |||
<tr> | |||
<td> Windows Vista | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 32-bit | |||
</td> | |||
<td> <b>manual</b>-see below - 2012-09-13 | |||
</td> | |||
<td> <b>Results</b> - yyyy-mm-dd | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 64-bit | |||
</td> | |||
<td> <b>manual</b>-see below - 2012-09-13 | |||
</td> | |||
<td> <b>Results</b> - yyyy-mm-dd | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.6 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.7 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.8 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-13 | |||
</td></tr></table> | |||
<p><i>Windows 8: manual spotcheck until automation in place</i> | |||
</p> | |||
<ul><li> RELEASETEST | |||
<ul><li> <b>PASS</b> - Firefox 16.0b1 en-US (partial) | |||
</li><li> <b>PASS</b> - Firefox 15.0b6 de (complete) | |||
</li></ul> | |||
</li><li> BETA | |||
<ul><li> <b>PASS</b> - Firefox 16.0b2 en-US (partial) | |||
</li><li> <b>PASS</b> - Firefox 15.0b6 de (complete) | |||
</li></ul> | |||
</li></ul> | |||
<dl><dt> Issues | |||
</dt></dl> | |||
<h3> Bug Triage 3 </h3> | |||
<ol><li> <a href="http://bit.ly/RquQy9">VERIFYME (3)</a> : verify the fix and do some exploratory testing around the issue | |||
</li><li> <a href="http://bit.ly/Ppe9E0">FIXED (184)</a> : triage these bugs and test to ensure the fix is effective | |||
</li><li> <a href="http://bit.ly/PMaYVx">QAWANTED (8)</a> : triage these bugs and test to fulfill the QA request | |||
</li><li> <a href="http://bit.ly/OtEqj1">UNCONFIRMED (10)</a> : triage these bugs for potential new regressions | |||
</li></ol> | |||
<dl><dt> Results | |||
</dt></dl> | |||
<ul><li> <a href="http://bit.ly/MXYdqS">VERIFIED</a>: 37/249 | |||
</li><li> <a href="http://bit.ly/POyxuA">WONTVERIFY</a>: 23/249 | |||
</li><li> <a href="http://bit.ly/O09WSK">HELPWANTED</a>: 5/249 | |||
</li></ul> | |||
<h2> Beta 4 </h2> | |||
<table class="querytable" style="width:30%"> | |||
<tr> | |||
<td style="width:10%"> | |||
</td> | |||
<td style="width:20%"> | |||
</td></tr> | |||
<tr> | |||
<td> <b>Builds</b> | |||
</td> | |||
<td> <a href="ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b4-candidates/build1/">FTP</a>, <a href="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b4-candidates/build1/">HTTP</a> | |||
</td></tr> | |||
<tr> | |||
<td> <b>ID</b> | |||
</td> | |||
<td> 20120919065210 | |||
</td></tr> | |||
<tr> | |||
<td> <b>Changeset</b> | |||
</td> | |||
<td> <a href="http://hg.mozilla.org/releases/mozilla-beta/rev/c3be659f6121">c3be659f6121</a> | |||
</td></tr></table> | |||
<dl><dt> Checklist | |||
</dt></dl> | |||
<ul><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Automated_Testing_4">Automated Testing</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Manual_Testing_4">Manual Testing</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Update_Testing_4">Update Testing</a> | |||
</li><li> <span class="fck_mw_template">{{ok|}}</span> <a href="Releases/Firefox 16/Test Plan#Bug_Triage_4">Bug Triage</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> Peer Sign-off | |||
</li><li> <span class="fck_mw_template">{{ | }}</span> Final sign-off sent and updates qualified on Beta channel | |||
</li></ul> | |||
<h3> Automated Testing 4 </h3> | |||
<p>The following are the detailed results of the automated Mozmill test runs. | |||
</p> | |||
<table class="querytable" style="width:50%"> | |||
<tr> | |||
<td class="header" style="width:20%"> Platform | |||
</td> | |||
<td class="header" style="width:30%"> RESULT: <a href="http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-09-18&to=2012-09-19">report</a> | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-19 | |||
</td></tr> | |||
<tr> | |||
<td> Windows Vista | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-19 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-19 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-19 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-19 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-19 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.6 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-19 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.7 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-19 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.8 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-19 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-19 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-19 | |||
</td></tr></table> | |||
<dl><dt> Test Failures (<a href="https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA">known bugs</a>) | |||
</dt></dl> | |||
<h3> Manual Testing 4 </h3> | |||
<table class="querytable"> | |||
<tr> | |||
<td class="header"> Operating System | |||
</td> | |||
<td class="header"> Tester | |||
</td> | |||
<td class="header"> Plug-in Compatibility | |||
</td> | |||
<td class="header"> Add-on Compatibility | |||
</td> | |||
<td class="header"> Web Compatibility | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> Vlad, Mihaela | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-21 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-21 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-21 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 | |||
</td> | |||
<td> Simona | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 | |||
</td> | |||
<td> Paul | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X | |||
</td> | |||
<td> VladG | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 | |||
</td> | |||
<td> Mihaela | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td></tr> | |||
<tr> | |||
<td class="header"> Operating System | |||
</td> | |||
<td class="header"> Tester | |||
</td> | |||
<td class="header"> Web Mail | |||
</td> | |||
<td class="header"> Mac 10.8 Signing | |||
</td> | |||
<td class="header"> Breakpad | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> Mihaela | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 | |||
</td> | |||
<td> Simona | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 | |||
</td> | |||
<td> Paul | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X | |||
</td> | |||
<td> VladG | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 | |||
</td> | |||
<td> Mihaela | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td></tr></table> | |||
<h4> Plug-in Compatibility 4 </h4> | |||
<p>Install the following plug-ins and go to websites which use them, looking for rendering issues, functional issues, performance issues, and crashes. | |||
</p> | |||
<ul><li> Adobe Flash Player | |||
<ul><li><a href="http://get.adobe.com/flashplayer/">11.4 Release</a> | |||
</li></ul> | |||
</li><li> <a href="http://www.microsoft.com/silverlight/">Microsoft Silverlight</a> | |||
</li><li> <a href="http://www.real.com/">RealPlayer</a> | |||
<ul><li> Real Player plugin crashed when opening pages with content managed by the plugin: <a href="http://www.slp3d2.com/launchsiteglobal/test.cfm">click</a>, <a href="http://cf.unc.edu/skillstest/take/realplayer.htm">click</a> - <span class="fck_mw_template">{{bug|667399}}</span> | |||
</li></ul> | |||
</li><li> <a href="http://www.apple.com/quicktime/download/">QuickTime</a> | |||
</li></ul> | |||
<h4> Add-on Compatibility 4 </h4> | |||
<p>Test the following websites for rendering issues, functional issues, performance issues, and crashes. Pay particular attention to any problems seen after updating Firefox with add-ons installed. | |||
</p> | |||
<ul><li> <a href="https://addons.mozilla.org/en-US/firefox/compatibility/15.0">Top AMO add-ons</a> | |||
</li><li> AVG toolbar (installed via AVG software) | |||
</li><li> Norton toolbar (installed via Norton software) | |||
</li><li> <a href="http://sg.toolbar.yahoo.com/">Yahoo toolbar</a> | |||
</li><li> <a href="http://bar.yandex.ru/en/">Yandex toolbar</a> | |||
</li><li> <a href="http://babylon-toolbar.software.informer.com/download/">Babylon toolbar</a> | |||
<ul><li> Babylon toolbar can't be installed on Firefox 16 - <span class="fck_mw_template">{{bug|721264}}</span> | |||
</li></ul> | |||
</li><li> <a href="http://lavasoft.com/">Ad-Aware Antivirus</a> | |||
</li></ul> | |||
<h4> Web Compatibility 4 </h4> | |||
<p>Test the following websites for rendering issues, functional issues, performance issues, and crashes | |||
</p> | |||
<ol><li> <a href="http://www.alexa.com/topsites">Overall Top 10</a> | |||
</li><li> <a href="http://www.alexa.com/topsites/countries">Top by Country</a> (pick a random site from three random countries) | |||
</li><li> <a href="http://www.alexa.com/topsites/category">Top by Category</a> (pick a random site from three random categories) | |||
</li><li> Webmail clients like gmail, hotmail, and yahoo mail (pay particular attention to any built in web messenger software) | |||
</li><li> Try Google's <a href="http://support.google.com/maps/bin/answer.py?hl=en&answer=1630790">MapsGL demo</a> | |||
</li><li> Try YouTube's <a href="http://www.youtube.com/html5">HTML5 demo</a> | |||
</li></ol> | |||
<h4> Web Mail 4 </h4> | |||
<p>Test the following websites for rendering issues, functional issues, performance issues, and crashes. In particular make sure email, attachments, web messenger, and video chat are working as expected. | |||
</p> | |||
<ol><li> <a href="https://mail.google.com/">Google Mail</a> | |||
</li><li> <a href="https://mail.yahoo.com/">Yahoo Mail</a> | |||
</li><li> <a href="https://www.hotmail.com">Hotmail</a> | |||
</li><li> <a href="https://www.outlook.com/">Outlook.com</a> | |||
</li></ol> | |||
<h4> Mac 10.8 Signing 4 </h4> | |||
<p>NOTE: Test needs to be run from a clean Mac OSX 10.8 install. Please keep a vanilla image handy so you can restore once completing this test (<a href="https://people.mozilla.com/~stmichaud/Gatekeeper/MLGatekeeper.html">more info</a>). | |||
</p> | |||
<ul><li> Download the .dmg file and drag the Firefox.app file to a location on your computer | |||
</li><li> Open a terminal window and cd to the folder Firefox.app is located | |||
</li><li> Run <i>codesign -vv Firefox.app</i> in terminal, output should contain <i>valid on disk</i> and <i>satisfies its Designated Requirement</i> | |||
</li><li> Run <i>xattr Firefox.app</i> in terminal, output should contain <i>com.apple.quarantine</i> | |||
</li><li> Double click the Firefox.app icon, you should see a warning about the program being provided by an <i>unknown developer</i> or having been <i>downloaded from the Internet</i> | |||
</li><li> Click "open" and browse to any website, Firefox should start and you should be able to browse | |||
</li><li> Quit Firefox and double click the Firefox.app icon again, you should <i>not</i> be reprompted for the warning in step 5, firefox should start up and browse without any issues | |||
</li></ul> | |||
<p>Spot check build after this test | |||
</p> | |||
<ul><li> Try all the menu item functions | |||
</li><li> Gestures | |||
</li><li> Plug-in compatibility | |||
</li><li> Top-site compatibility | |||
</li><li> Add-on compatibility | |||
</li><li> Software update | |||
</li></ul> | |||
<h4> Breakpad End-to-End 4 </h4> | |||
<p>Test the end-to-end functionality in the crash reporting feature, across operating systems. | |||
</p> | |||
<ul><li> Use the <a href="http://ted.mielczarek.org/mozilla/crashme.html">Crash Me add-on</a> to trigger a crash | |||
</li><li> Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc | |||
</li><li> Repeatedly crash Firefox and submit your reports | |||
</li><li> Verify that your session restores properly | |||
</li><li> Verify your crash reports are submitted properly by going through about:crashes and clicking the reports | |||
</li><li> Try adding comments to your crash report and verify the crash-stats.com report shows the comment | |||
</li></ul> | |||
<h3> Update Testing 4 </h3> | |||
<ul><li> Partial: 16.0b3 | |||
</li><li> Complete: 16.0b2 and earlier | |||
</li></ul> | |||
<table class="querytable" style="width:60%"> | |||
; | <tr> | ||
< | <td class="header"> Platform | ||
{ | </td> | ||
<td class="header"> Releasetest: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-09-18&to=2012-09-19&target=16.0">report</a> | |||
} | </td> | ||
</ | <td class="header"> Beta: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-09-18&to=2012-09-19&target=16.0">report</a> | ||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-09-19 | |||
</td></tr> | |||
<tr> | |||
<td> Windows Vista | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-09-19 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-09-19 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-09-19 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 32-bit | |||
</td> | |||
<td> <b>manual</b>-see below - 2012-09-19 | |||
</td> | |||
<td> <b>Results</b> - yyyy-mm-dd | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 64-bit | |||
</td> | |||
<td> <b>manual</b>-see below - 2012-09-19 | |||
</td> | |||
<td> <b>Results</b> - yyyy-mm-dd | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.6 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-09-19 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.7 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-09-19 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.8 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-09-19 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-09-19 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-20 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-09-19 | |||
</td></tr></table> | |||
<p><i>Windows 8: manual spotcheck until automation in place</i> | |||
</p> | |||
<ul><li> RELEASETEST | |||
<ul><li> <b>PASS</b> - Firefox 16.0b3 en-US (partial) | |||
</li><li> <b>PASS</b> - Firefox 15.0b6 de (complete) | |||
</li></ul> | |||
</li><li> BETA | |||
<ul><li> <b>RESULT</b> - Firefox 16.0b3 en-US (partial) | |||
</li><li> <b>RESULT</b> - Firefox 15.0b6 de (complete) | |||
</li></ul> | |||
</li></ul> | |||
<dl><dt> Issues | |||
</dt></dl> | |||
<h3> Bug Triage 4 </h3> | |||
<ol><li> <a href="http://bit.ly/RquQy9">VERIFYME (?)</a> : verify the fix and do some exploratory testing around the issue | |||
</li><li> <a href="http://bit.ly/Ppe9E0">FIXED (?)</a> : triage these bugs and test to ensure the fix is effective | |||
</li><li> <a href="http://bit.ly/PMaYVx">QAWANTED (?)</a> : triage these bugs and test to fulfill the QA request | |||
</li><li> <a href="http://bit.ly/OtEqj1">UNCONFIRMED (?)</a> : triage these bugs for potential new regressions | |||
</li></ol> | |||
<dl><dt> Results | |||
</dt></dl> | |||
<ul><li> <a href="http://bit.ly/MXYdqS">VERIFIED</a>: ratios | |||
</li><li> <a href="http://bit.ly/POyxuA">WONTVERIFY</a>: ratios | |||
</li><li> <a href="http://bit.ly/O09WSK">HELPWANTED</a>: ratios | |||
</li></ul> | |||
<p>Verification <a href="https://docs.google.com/spreadsheet/ccc?key=0AoLS2jzcUzsHdEFLcDlFa1ppTWZXRDBvZ3Btamk2MVE#gid=0">Spreadsheet</a> | |||
</p><p><br /> | |||
</p> | |||
<h2> Beta 5 </h2> | |||
<table class="querytable" style="width:30%"> | |||
<tr> | |||
<td style="width:10%"> | |||
</td> | |||
<td style="width:20%"> | |||
</td></tr> | |||
<tr> | |||
<td> <b>Builds</b> | |||
</td> | |||
<td> <a href="ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b5-candidates/build1/">FTP</a>, <a href="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b5-candidates/build1/">HTTP</a> | |||
</td></tr> | |||
<tr> | |||
<td> <b>ID</b> | |||
</td> | |||
<td> 20120925201946 | |||
</td></tr> | |||
<tr> | |||
<td> <b>Changeset</b> | |||
</td> | |||
<td> <a href="https://hg.mozilla.org/releases/mozilla-beta/rev/7c0af9b7ed61">7c0af9b7ed61</a> (<a href="http://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=FIREFOX_16_0b4_BUILD1&tochange=FIREFOX_16_0b5_BUILD1">pushlog</a>) | |||
</td></tr></table> | |||
<dl><dt> Checklist | |||
</dt></dl> | |||
<ul><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Automated_Testing_5">Automated Testing</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Manual_Testing_5">Manual Testing</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Update_Testing_5">Update Testing</a> | |||
</li><li> <span class="fck_mw_template">{{ok|}}</span> <a href="Releases/Firefox 16/Test Plan#Bug_Triage_5">Bug Triage</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> Peer Sign-off | |||
</li><li> <span class="fck_mw_template">{{ | }}</span> Final sign-off sent and updates qualified on Beta channel | |||
</li></ul> | |||
<h3> Automated Testing 5 </h3> | |||
<p>The following are the detailed results of the automated Mozmill test runs. | |||
</p> | |||
<table class="querytable" style="width:50%"> | |||
=== | <tr> | ||
<td class="header" style="width:20%"> Platform | |||
</td> | |||
<td class="header" style="width:30%"> RESULT: <a href="http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-09-25&to=2012-09-26">report</a> | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td></tr> | |||
<tr> | |||
<td> Windows Vista | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.6 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.7 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.8 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td></tr></table> | |||
<dl><dt> Test Failures (<a href="https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA">known bugs</a>) | |||
</dt></dl> | |||
<h3> Manual Testing 5 </h3> | |||
<table class="querytable"> | |||
<tr> | |||
<td class="header"> Operating System | |||
</td> | |||
<td class="header"> Tester | |||
</td> | |||
<td class="header"> Plug-in Compatibility | |||
</td> | |||
<td class="header"> Add-on Compatibility | |||
</td> | |||
<td class="header"> Web Compatibility | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> Simona | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 | |||
</td> | |||
<td> Paul | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 | |||
</td> | |||
<td> Mihaela | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X | |||
</td> | |||
<td> VladG | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 | |||
</td> | |||
<td> Ioana | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-27 | |||
</td></tr> | |||
<tr> | |||
<td class="header"> Operating System | |||
</td> | |||
<td class="header"> Tester | |||
</td> | |||
<td class="header"> Web Mail | |||
</td> | |||
<td class="header"> Mac 10.8 Signing | |||
</td> | |||
<td class="header"> Breakpad | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> Simona | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 | |||
</td> | |||
<td> Paul | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 | |||
</td> | |||
<td> Mihaela | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-27 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X | |||
</td> | |||
<td> VladG | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-27 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-27 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 | |||
</td> | |||
<td> Ioana | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-27 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-27 | |||
</td></tr></table> | |||
<h4> Plug-in Compatibility 5 </h4> | |||
<p>Install the following plug-ins and go to websites which use them, looking for rendering issues, functional issues, performance issues, and crashes. | |||
</p> | |||
<ul><li> Adobe Flash Player | |||
<ul><li><a href="http://get.adobe.com/flashplayer/">11.4 Release</a> | |||
</li></ul> | |||
</li><li> <a href="http://www.microsoft.com/silverlight/">Microsoft Silverlight</a> | |||
</li><li> <a href="http://www.real.com/">RealPlayer</a> | |||
<ul><li> Real Player plugin crashed when opening pages with content managed by the plugin: <a href="http://www.slp3d2.com/launchsiteglobal/test.cfm">click</a>, <a href="http://cf.unc.edu/skillstest/take/realplayer.htm">click</a> - <span class="fck_mw_template">{{bug|667399}}</span> | |||
</li></ul> | |||
</li><li> <a href="http://www.apple.com/quicktime/download/">QuickTime</a> | |||
</li></ul> | |||
<h4> Add-on Compatibility 5 </h4> | |||
<p>Test the following websites for rendering issues, functional issues, performance issues, and crashes. Pay particular attention to any problems seen after updating Firefox with add-ons installed. | |||
</p> | |||
<ul><li> <a href="https://addons.mozilla.org/en-US/firefox/compatibility/16.0">Top AMO add-ons</a> | |||
</li><li> AVG toolbar (installed via AVG software) | |||
</li><li> Norton toolbar (installed via Norton software) | |||
</li><li> <a href="http://sg.toolbar.yahoo.com/">Yahoo toolbar</a> | |||
</li><li> <a href="http://bar.yandex.ru/en/">Yandex toolbar</a> | |||
</li><li> <a href="http://babylon-toolbar.software.informer.com/download/">Babylon toolbar</a> | |||
<ul><li> Babylon toolbar can't be installed on Firefox 16 - <span class="fck_mw_template">{{bug|721264}}</span> | |||
</li></ul> | |||
</li><li> <a href="http://lavasoft.com/">Ad-Aware Antivirus</a> | |||
</li></ul> | |||
<h4> Web Compatibility 5 </h4> | |||
<p>Test the following websites for rendering issues, functional issues, performance issues, and crashes | |||
</p> | |||
<ol><li> <a href="http://www.alexa.com/topsites">Overall Top 10</a> | |||
</li><li> <a href="http://www.alexa.com/topsites/countries">Top by Country</a> (pick a random site from three random countries) | |||
</li><li> <a href="http://www.alexa.com/topsites/category">Top by Category</a> (pick a random site from three random categories) | |||
</li><li> Webmail clients like gmail, hotmail, and yahoo mail (pay particular attention to any built in web messenger software) | |||
</li><li> Try Google's <a href="http://support.google.com/maps/bin/answer.py?hl=en&answer=1630790">MapsGL demo</a> | |||
</li><li> Try YouTube's <a href="http://www.youtube.com/html5">HTML5 demo</a> | |||
</li></ol> | |||
<h4> Web Mail 5 </h4> | |||
<p>Test the following websites for rendering issues, functional issues, performance issues, and crashes. In particular make sure email, attachments, web messenger, and video chat are working as expected. | |||
</p> | |||
<ol><li> <a href="https://mail.google.com/">Google Mail</a> | |||
</li><li> <a href="https://mail.yahoo.com/">Yahoo Mail</a> | |||
</li><li> <a href="https://www.hotmail.com">Hotmail</a> | |||
</li><li> <a href="https://www.outlook.com/">Outlook.com</a> | |||
</li></ol> | |||
<h4> Mac 10.8 Signing 5 </h4> | |||
<p>NOTE: Test needs to be run from a clean Mac OSX 10.8 install. Please keep a vanilla image handy so you can restore once completing this test (<a href="https://people.mozilla.com/~stmichaud/Gatekeeper/MLGatekeeper.html">more info</a>). | |||
</p> | |||
<ul><li> Download the .dmg file and drag the Firefox.app file to a location on your computer | |||
</li><li> Open a terminal window and cd to the folder Firefox.app is located | |||
</li><li> Run <i>codesign -vv Firefox.app</i> in terminal, output should contain <i>valid on disk</i> and <i>satisfies its Designated Requirement</i> | |||
</li><li> Run <i>xattr Firefox.app</i> in terminal, output should contain <i>com.apple.quarantine</i> | |||
</li><li> Double click the Firefox.app icon, you should see a warning about the program being provided by an <i>unknown developer</i> or having been <i>downloaded from the Internet</i> | |||
</li><li> Click "open" and browse to any website, Firefox should start and you should be able to browse | |||
</li><li> Quit Firefox and double click the Firefox.app icon again, you should <i>not</i> be reprompted for the warning in step 5, firefox should start up and browse without any issues | |||
</li></ul> | |||
<p>Spot check build after this test | |||
</p> | |||
<ul><li> Try all the menu item functions | |||
</li><li> Gestures | |||
</li><li> Plug-in compatibility | |||
</li><li> Top-site compatibility | |||
</li><li> Add-on compatibility | |||
</li><li> Software update | |||
</li></ul> | |||
<h4> Breakpad End-to-End 5 </h4> | |||
<p>Test the end-to-end functionality in the crash reporting feature, across operating systems. | |||
</p> | |||
<ul><li> Use the <a href="http://ted.mielczarek.org/mozilla/crashme.html">Crash Me add-on</a> to trigger a crash | |||
</li><li> Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc | |||
</li><li> Repeatedly crash Firefox and submit your reports | |||
</li><li> Verify that your session restores properly | |||
</li><li> Verify your crash reports are submitted properly by going through about:crashes and clicking the reports | |||
</li><li> Try adding comments to your crash report and verify the crash-stats.com report shows the comment | |||
</li></ul> | |||
<h4> Windows 8 </h4> | |||
<p>Test Firefox integration with Windows 8 - <a href="https://etherpad.mozilla.org/Win-8-Classic-Test-Plan">etherpad</a>. | |||
</p> | |||
<h3> Update Testing 5 </h3> | |||
<ul><li> Partial: 16.0b4 | |||
</li><li> Complete: 16.0b3 and earlier | |||
</li></ul> | |||
<table class="querytable" style="width:60%"> | |||
; | <tr> | ||
<td class="header"> Platform | |||
# | </td> | ||
<td class="header"> Releasetest: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-09-25&to=2012-09-26&target=16.0">report</a> | |||
</td> | |||
<td class="header"> Beta: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-09-25&to=2012-09-26&target=16.0">report</a> | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-09-26 | |||
</td></tr> | |||
<tr> | |||
<td> Windows Vista | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-09-26 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-09-26 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-09-26 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 32-bit | |||
</td> | |||
<td> <b>manual</b>-see below - 2012-09-19 | |||
</td> | |||
<td> <b>Results</b> - yyyy-mm-dd | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 64-bit | |||
</td> | |||
<td> <b>manual</b>-see below - 2012-09-19 | |||
</td> | |||
<td> <b>Results</b> - yyyy-mm-dd | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.6 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-09-26 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.7 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-09-26 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.8 | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-09-26 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-09-26 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-09-26 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-09-26 | |||
</td></tr></table> | |||
<p><i>Windows 8: manual spotcheck until automation in place</i> | |||
</p> | |||
<ul><li> RELEASETEST | |||
<ul><li> <b>RESULT</b> - Firefox 16.0b4 en-US (partial) | |||
</li><li> <b>RESULT</b> - Firefox 15.0b6 de (complete) | |||
</li></ul> | |||
</li><li> BETA | |||
<ul><li> <b>RESULT</b> - Firefox 16.0b4 en-US (partial) | |||
</li><li> <b>RESULT</b> - Firefox 15.0b6 de (complete) | |||
</li></ul> | |||
</li></ul> | |||
<dl><dt> Issues | |||
</dt></dl> | |||
<h3> Bug Triage 5 </h3> | |||
<ol><li> <a href="http://bit.ly/RquQy9">VERIFYME (19)</a> : verify the fix and do some exploratory testing around the issue | |||
</li><li> <a href="http://bit.ly/Ppe9E0">FIXED (184)</a> : triage these bugs and test to ensure the fix is effective | |||
</li><li> <a href="http://bit.ly/PMaYVx">QAWANTED (6)</a> : triage these bugs and test to fulfill the QA request | |||
</li><li> <a href="http://bit.ly/OtEqj1">UNCONFIRMED (14)</a> : triage these bugs for potential new regressions | |||
</li></ol> | |||
<dl><dt> Results | |||
</dt></dl> | |||
<ul><li> <a href="http://bit.ly/MXYdqS">VERIFIED</a>: 81/296 | |||
</li><li> <a href="http://bit.ly/POyxuA">WONTVERIFY</a>: 30/296 | |||
</li><li> <a href="http://bit.ly/O09WSK">HELPWANTED</a>: 5/296 | |||
</li></ul> | |||
<h2> Beta 6 </h2> | |||
<table class="querytable" style="width:30%"> | |||
<tr> | |||
<td style="width:10%"> | |||
</td> | |||
<td style="width:20%"> | |||
</td></tr> | |||
<tr> | |||
<td> <b>Builds</b> | |||
</td> | |||
<td> <a href="ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b6-candidates/build1/">FTP</a>, <a href="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b6-candidates/build1/">HTTP</a> | |||
</td></tr> | |||
<tr> | |||
<td> <b>ID</b> | |||
</td> | |||
<td> 20121002073616 | |||
</td></tr> | |||
<tr> | |||
<td> <b>Changeset</b> | |||
</td> | |||
<td> <a href="https://hg.mozilla.org/releases/mozilla-beta/rev/8a8a932fbef4">8a8a932fbef4</a> (<a href="http://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=FIREFOX_16_0b5_BUILD1&tochange=FIREFOX_16_0b6_BUILD1">pushlog</a>) | |||
</td></tr></table> | |||
<dl><dt> Checklist | |||
</dt></dl> | |||
<ul><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Automated_Testing_6">Automated Testing</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Manual_Testing_6">Manual Testing</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Update_Testing_6">Update Testing</a> | |||
</li><li> <span class="fck_mw_template">{{ok|}}</span> <a href="Releases/Firefox 16/Test Plan#Bug_Triage_6">Bug Triage</a> | |||
</li><li> <span class="fck_mw_template">{{ok|}}</span> Peer Sign-off | |||
</li><li> <span class="fck_mw_template">{{ | }}</span> Final sign-off sent and updates qualified on Beta channel | |||
</li></ul> | |||
<h3> Automated Testing 6 </h3> | |||
<p>The following are the detailed results of the automated Mozmill test runs. | |||
</p> | |||
<table class="querytable" style="width:50%"> | |||
; | <tr> | ||
<td class="header" style="width:20%"> Platform | |||
</td> | |||
<td class="header" style="width:30%"> RESULT: <a href="http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-10-01&to=2012-10-02">report</a> | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td></tr> | |||
<tr> | |||
<td> Windows Vista | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.6 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.7 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.8 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td></tr></table> | |||
<dl><dt> Test Failures (<a href="https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA">known bugs</a>) | |||
</dt></dl> | |||
<h3> Manual Testing 6 </h3> | |||
<table class="querytable"> | |||
<tr> | |||
<td class="header"> Operating System | |||
</td> | |||
<td class="header"> Tester | |||
</td> | |||
<td class="header"> Plug-in Compatibility | |||
</td> | |||
<td class="header"> Add-on Compatibility | |||
</td> | |||
<td class="header"> Web Compatibility | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> Paul/Virgil | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 | |||
</td> | |||
<td> Ioana | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 | |||
</td> | |||
<td> Simona | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X | |||
</td> | |||
<td> VladG | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-03 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-03 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-03 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 | |||
</td> | |||
<td> Mihaela | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td></tr> | |||
<tr> | |||
<td class="header"> Operating System | |||
</td> | |||
<td class="header"> Tester | |||
</td> | |||
<td class="header"> Web Mail | |||
</td> | |||
<td class="header"> Mac 10.8 Signing | |||
</td> | |||
<td class="header"> Breakpad | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> Paul/Virgil | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 | |||
</td> | |||
<td> Ioana/Paul | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 | |||
</td> | |||
<td> Simona | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X | |||
</td> | |||
<td> VladG | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-03 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-03 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-03 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 | |||
</td> | |||
<td> Mihaela | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td></tr></table> | |||
<h4> Plug-in Compatibility 6 </h4> | |||
<p>Install the following plug-ins and go to websites which use them, looking for rendering issues, functional issues, performance issues, and crashes. | |||
</p> | |||
<ul><li> Adobe Flash Player | |||
<ul><li><a href="http://get.adobe.com/flashplayer/">11.4 Release</a> | |||
</li></ul> | |||
</li><li> <a href="http://www.microsoft.com/silverlight/">Microsoft Silverlight</a> | |||
</li><li> <a href="http://www.real.com/">RealPlayer</a> | |||
<ul><li> Real Player plugin crashed when opening pages with content managed by the plugin: <a href="http://www.slp3d2.com/launchsiteglobal/test.cfm">click</a>, <a href="http://cf.unc.edu/skillstest/take/realplayer.htm">click</a> - <span class="fck_mw_template">{{bug|667399}}</span> | |||
</li></ul> | |||
</li><li> <a href="http://www.apple.com/quicktime/download/">QuickTime</a> | |||
</li></ul> | |||
<h4> Add-on Compatibility 6 </h4> | |||
<p>Test the following websites for rendering issues, functional issues, performance issues, and crashes. Pay particular attention to any problems seen after updating Firefox with add-ons installed. | |||
</p> | |||
<ul><li> <a href="https://addons.mozilla.org/en-US/firefox/compatibility/16.0">Top AMO add-ons</a> | |||
</li><li> AVG toolbar (installed via AVG software) | |||
</li><li> Norton toolbar (installed via Norton software) | |||
</li><li> <a href="http://sg.toolbar.yahoo.com/">Yahoo toolbar</a> | |||
</li><li> <a href="http://bar.yandex.ru/en/">Yandex toolbar</a> | |||
</li><li> <a href="http://babylon-toolbar.software.informer.com/download/">Babylon toolbar</a> | |||
</li><li> <a href="http://lavasoft.com/">Ad-Aware Antivirus</a> | |||
</li></ul> | |||
<h4> Web Compatibility 6 </h4> | |||
<p>Test the following websites for rendering issues, functional issues, performance issues, and crashes | |||
</p> | |||
<ol><li> <a href="http://www.alexa.com/topsites">Overall Top 10</a> | |||
</li><li> <a href="http://www.alexa.com/topsites/countries">Top by Country</a> (pick a random site from three random countries) | |||
</li><li> <a href="http://www.alexa.com/topsites/category">Top by Category</a> (pick a random site from three random categories) | |||
</li><li> Webmail clients like gmail, hotmail, and yahoo mail (pay particular attention to any built in web messenger software) | |||
</li><li> Try Google's <a href="http://support.google.com/maps/bin/answer.py?hl=en&answer=1630790">MapsGL demo</a> | |||
</li><li> Try YouTube's <a href="http://www.youtube.com/html5">HTML5 demo</a> | |||
</li></ol> | |||
<h4> Web Mail 6 </h4> | |||
<p>Test the following websites for rendering issues, functional issues, performance issues, and crashes. In particular make sure email, attachments, web messenger, and video chat are working as expected. | |||
</p> | |||
<ol><li> <a href="https://mail.google.com/">Google Mail</a> | |||
</li><li> <a href="https://mail.yahoo.com/">Yahoo Mail</a> | |||
</li><li> <a href="https://www.hotmail.com">Hotmail</a> | |||
</li><li> <a href="https://www.outlook.com/">Outlook.com</a> | |||
</li></ol> | |||
<h4> Mac 10.8 Signing 6 </h4> | |||
<p>NOTE: Test needs to be run from a clean Mac OSX 10.8 install. Please keep a vanilla image handy so you can restore once completing this test (<a href="https://people.mozilla.com/~stmichaud/Gatekeeper/MLGatekeeper.html">more info</a>). | |||
</p> | |||
<ul><li> Download the .dmg file and drag the Firefox.app file to a location on your computer | |||
</li><li> Open a terminal window and cd to the folder Firefox.app is located | |||
</li><li> Run <i>codesign -vv Firefox.app</i> in terminal, output should contain <i>valid on disk</i> and <i>satisfies its Designated Requirement</i> | |||
</li><li> Run <i>xattr Firefox.app</i> in terminal, output should contain <i>com.apple.quarantine</i> | |||
</li><li> Double click the Firefox.app icon, you should see a warning about the program being provided by an <i>unknown developer</i> or having been <i>downloaded from the Internet</i> | |||
</li><li> Click "open" and browse to any website, Firefox should start and you should be able to browse | |||
</li><li> Quit Firefox and double click the Firefox.app icon again, you should <i>not</i> be reprompted for the warning in step 5, firefox should start up and browse without any issues | |||
</li></ul> | |||
<p>Spot check build after this test | |||
</p> | |||
<ul><li> Try all the menu item functions | |||
</li><li> Gestures | |||
</li><li> Plug-in compatibility | |||
</li><li> Top-site compatibility | |||
</li><li> Add-on compatibility | |||
</li><li> Software update | |||
</li></ul> | |||
<h4> Breakpad End-to-End 6 </h4> | |||
<p>Test the end-to-end functionality in the crash reporting feature, across operating systems. | |||
</p> | |||
<ul><li> Use the <a href="http://ted.mielczarek.org/mozilla/crashme.html">Crash Me add-on</a> to trigger a crash | |||
</li><li> Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc | |||
</li><li> Repeatedly crash Firefox and submit your reports | |||
</li><li> Verify that your session restores properly | |||
</li><li> Verify your crash reports are submitted properly by going through about:crashes and clicking the reports | |||
</li><li> Try adding comments to your crash report and verify the crash-stats.com report shows the comment | |||
</li></ul> | |||
<h3> Update Testing 6 </h3> | |||
<ul><li> Partial: 16.0b5 | |||
</li><li> Complete: 16.0b4 and earlier | |||
</li></ul> | |||
<table class="querytable" style="width:60%"> | |||
<tr> | |||
<td class="header"> Platform | |||
</td> | |||
<td class="header"> Releasetest: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-10-02&to=2012-10-03&target=16.0">report</a> | |||
</td> | |||
<td class="header"> Beta: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-10-02&to=2012-10-03&target=16.0">report</a> | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-10-02 | |||
</td></tr> | |||
<tr> | |||
<td> Windows Vista | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-10-02 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-10-02 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-10-02 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 32-bit | |||
</td> | |||
<td> <b>manual</b>-see below - 2012-10-02 | |||
</td> | |||
<td> <b>Results</b> - yyyy-mm-dd | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 64-bit | |||
</td> | |||
<td> <b>manual</b>-see below - 2012-10-02 | |||
</td> | |||
<td> <b>Results</b> - yyyy-mm-dd | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.6 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-10-02 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.7 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-10-02 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.8 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-10-02 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-10-02 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-02 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-10-02 | |||
</td></tr></table> | |||
<p><i>Windows 8: manual spotcheck until automation in place</i> | |||
</p> | |||
<ul><li> RELEASETEST | |||
<ul><li> <b>RESULT</b> - Firefox 16.0b5 en-US (partial) | |||
</li><li> <b>RESULT</b> - Firefox 15.0b6 de (complete) | |||
</li></ul> | |||
</li><li> BETA | |||
<ul><li> <b>RESULT</b> - Firefox 16.0b5 en-US (partial) | |||
</li><li> <b>RESULT</b> - Firefox 15.0b6 de (complete) | |||
</li></ul> | |||
</li></ul> | |||
<dl><dt> Issues | |||
</dt></dl> | |||
<h3> Bug Triage 6 </h3> | |||
<ol><li> <a href="http://bit.ly/RquQy9">VERIFYME (10)</a> : verify the fix and do some exploratory testing around the issue | |||
</li><li> <a href="http://bit.ly/Ppe9E0">FIXED (181)</a> : triage these bugs and test to ensure the fix is effective | |||
</li><li> <a href="http://bit.ly/PMaYVx">QAWANTED (4)</a> : triage these bugs and test to fulfill the QA request | |||
</li><li> <a href="http://bit.ly/OtEqj1">UNCONFIRMED (18)</a> : triage these bugs for potential new regressions | |||
</li></ol> | |||
<dl><dt> Results | |||
</dt></dl> | |||
<ul><li> <a href="http://bit.ly/MXYdqS">VERIFIED</a>: 87/303 | |||
</li><li> <a href="http://bit.ly/POyxuA">WONTVERIFY</a>: 31/303 | |||
</li><li> <a href="http://bit.ly/O09WSK">HELPWANTED</a>: 4/303 | |||
</li></ul> | |||
<h2> Release Candidate </h2> | |||
<table class="querytable" style="width:30%"> | |||
<tr> | |||
<td style="width:10%"> | |||
</td> | |||
<td style="width:20%"> | |||
</td></tr> | |||
<tr> | |||
<td> <b>Builds</b> | |||
</td> | |||
<td> <a href="ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0-candidates/build1/">FTP</a>, <a href="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0-candidates/build1/">HTTP</a> | |||
</td></tr> | |||
<tr> | |||
<td> <b>ID</b> | |||
</td> | |||
<td> 20121005155445 | |||
</td></tr> | |||
<tr> | |||
<td> <b>Changeset</b> | |||
</td> | |||
<td> <a href="http://hg.mozilla.org/releases/mozilla-release/rev/10fe550fadc6">10fe550fadc6</a> | |||
</td></tr></table> | |||
<dl><dt> Checklist | |||
</dt></dl> | |||
<ul><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Automated_Testing_RC">Automated Testing</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Manual_Testing_RC">Manual Testing</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Update_Testing_RC">Update Testing</a> | |||
</li><li> <span class="fck_mw_template">{{ok|}}</span> <a href="Releases/Firefox 16/Test Plan#Bug_Triage_RC">Bug Triage</a> | |||
</li><li> <span class="fck_mw_template">{{|}}</span> Peer Sign-off | |||
</li><li> <span class="fck_mw_template">{{ | }}</span> Final sign-off sent and updates qualified on Release channel | |||
</li></ul> | |||
<h3> Automated Testing RC</h3> | |||
<p>The following are the detailed results of the automated Mozmill test runs. | |||
</p> | |||
<table class="querytable" style="width:50%"> | |||
=== | <tr> | ||
<td class="header" style="width:20%"> Platform | |||
</td> | |||
<td class="header" style="width:30%"> RESULT: <a href="http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-10-05&to=2012-10-09">report</a> | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-08 | |||
</td></tr> | |||
<tr> | |||
<td> Windows Vista | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-08 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-08 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-08 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-08 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-08 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.6 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-08 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.7 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-08 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.8 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-08 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-08 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-08 | |||
</td></tr></table> | |||
<dl><dt> Test Failures (<a href="https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA">known bugs</a>) | |||
</dt></dl> | |||
<h3> Manual Testing RC </h3> | |||
<table class="querytable"> | |||
<tr> | |||
<td class="header"> Operating System | |||
</td> | |||
<td class="header"> Tester | |||
</td> | |||
<td class="header"> Plug-in Compatibility | |||
</td> | |||
<td class="header"> Add-on Compatibility | |||
</td> | |||
<td class="header"> Web Compatibility | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> Ioana | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-09 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-09 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-09 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 | |||
</td> | |||
<td> Simona | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-08 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-09 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-09 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 | |||
</td> | |||
<td> Mihaela | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-09 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-09 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-09 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X | |||
</td> | |||
<td> VladG | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-09 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-09 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-09 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 | |||
</td> | |||
<td> Virgil | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-08 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-09 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-08 | |||
</td></tr> | |||
<tr> | |||
<td class="header"> Operating System | |||
</td> | |||
<td class="header"> Tester | |||
</td> | |||
<td class="header"> Web Mail | |||
</td> | |||
<td class="header"> Mac 10.8 Signing | |||
</td> | |||
<td class="header"> Breakpad | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> Paul | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-09 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-09 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 | |||
</td> | |||
<td> Simona | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-09 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-09 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 | |||
</td> | |||
<td> Mihaela | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-09 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-09 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X | |||
</td> | |||
<td> VladG | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-09 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-08 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-09 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 | |||
</td> | |||
<td> Virgil | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-08 | |||
</td> | |||
<td> <b>N/A</b> | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-09 | |||
</td></tr></table> | |||
<h4> Plug-in Compatibility </h4> | |||
<p>Install the following plug-ins and go to websites which use them, looking for rendering issues, functional issues, performance issues, and crashes. | |||
</p> | |||
<ul><li> Adobe Flash Player | |||
<ul><li><a href="http://get.adobe.com/flashplayer/">11.4 Release</a> | |||
</li></ul> | |||
</li><li> <a href="http://www.microsoft.com/silverlight/">Microsoft Silverlight</a> | |||
</li><li> <a href="http://www.real.com/">RealPlayer</a> | |||
<ul><li> Real Player plugin crashed when opening pages with content managed by the plugin: <a href="http://www.slp3d2.com/launchsiteglobal/test.cfm">click</a>, <a href="http://cf.unc.edu/skillstest/take/realplayer.htm">click</a> - <span class="fck_mw_template">{{bug|667399}}</span> | |||
</li></ul> | |||
</li><li> <a href="http://www.apple.com/quicktime/download/">QuickTime</a> | |||
</li></ul> | |||
<h4> Add-on Compatibility </h4> | |||
<p>Test the following websites for rendering issues, functional issues, performance issues, and crashes. Pay particular attention to any problems seen after updating Firefox with add-ons installed. | |||
</p> | |||
<ul><li> <a href="https://addons.mozilla.org/en-US/firefox/compatibility/16.0">Top AMO add-ons</a> | |||
</li><li> AVG toolbar (installed via AVG software) | |||
</li><li> Norton toolbar (installed via Norton software) | |||
</li><li> <a href="http://sg.toolbar.yahoo.com/">Yahoo toolbar</a> | |||
</li><li> <a href="http://bar.yandex.ru/en/">Yandex toolbar</a> | |||
</li><li> <a href="http://babylon-toolbar.software.informer.com/download/">Babylon toolbar</a> | |||
</li><li> <a href="http://lavasoft.com/">Ad-Aware Antivirus</a> | |||
</li></ul> | |||
<h4> Web Compatibility </h4> | |||
<p>Test the following websites for rendering issues, functional issues, performance issues, and crashes | |||
</p> | |||
<ol><li> <a href="http://www.alexa.com/topsites">Overall Top 10</a> | |||
</li><li> <a href="http://www.alexa.com/topsites/countries">Top by Country</a> (pick a random site from three random countries) | |||
</li><li> <a href="http://www.alexa.com/topsites/category">Top by Category</a> (pick a random site from three random categories) | |||
</li><li> Webmail clients like gmail, hotmail, and yahoo mail (pay particular attention to any built in web messenger software) | |||
</li><li> Try Google's <a href="http://support.google.com/maps/bin/answer.py?hl=en&answer=1630790">MapsGL demo</a> | |||
</li><li> Try YouTube's <a href="http://www.youtube.com/html5">HTML5 demo</a> | |||
</li></ol> | |||
<h4> Web Mail </h4> | |||
<p>Test the following websites for rendering issues, functional issues, performance issues, and crashes. In particular make sure email, attachments, web messenger, and video chat are working as expected. | |||
</p> | |||
<ol><li> <a href="https://mail.google.com/">Google Mail</a> | |||
</li><li> <a href="https://mail.yahoo.com/">Yahoo Mail</a> | |||
</li><li> <a href="https://www.hotmail.com">Hotmail</a> | |||
</li><li> <a href="https://www.outlook.com/">Outlook.com</a> | |||
</li></ol> | |||
<h4> Mac 10.8 Signing </h4> | |||
<p>NOTE: Test needs to be run from a clean Mac OSX 10.8 install. Please keep a vanilla image handy so you can restore once completing this test (<a href="https://people.mozilla.com/~stmichaud/Gatekeeper/MLGatekeeper.html">more info</a>). | |||
</p> | |||
<ul><li> Download the .dmg file and drag the Firefox.app file to a location on your computer | |||
</li><li> Open a terminal window and cd to the folder Firefox.app is located | |||
</li><li> Run <i>codesign -vv Firefox.app</i> in terminal, output should contain <i>valid on disk</i> and <i>satisfies its Designated Requirement</i> | |||
</li><li> Run <i>xattr Firefox.app</i> in terminal, output should contain <i>com.apple.quarantine</i> | |||
</li><li> Double click the Firefox.app icon, you should see a warning about the program being provided by an <i>unknown developer</i> or having been <i>downloaded from the Internet</i> | |||
</li><li> Click "open" and browse to any website, Firefox should start and you should be able to browse | |||
</li><li> Quit Firefox and double click the Firefox.app icon again, you should <i>not</i> be reprompted for the warning in step 5, firefox should start up and browse without any issues | |||
</li></ul> | |||
<p>Spot check build after this test | |||
</p> | |||
<ul><li> Try all the menu item functions | |||
</li><li> Gestures | |||
</li><li> Plug-in compatibility | |||
</li><li> Top-site compatibility | |||
</li><li> Add-on compatibility | |||
</li><li> Software update | |||
</li></ul> | |||
<h4> Breakpad End-to-End </h4> | |||
<p>Test the end-to-end functionality in the crash reporting feature, across operating systems. | |||
</p> | |||
<ul><li> Use the <a href="http://ted.mielczarek.org/mozilla/crashme.html">Crash Me add-on</a> to trigger a crash | |||
</li><li> Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc | |||
</li><li> Repeatedly crash Firefox and submit your reports | |||
</li><li> Verify that your session restores properly | |||
</li><li> Verify your crash reports are submitted properly by going through about:crashes and clicking the reports | |||
</li><li> Try adding comments to your crash report and verify the crash-stats.com report shows the comment | |||
</li></ul> | |||
<h3> Update Testing RC </h3> | |||
<ul><li> Partial: 15.0.1 | |||
</li><li> Complete: 12.0 and earlier | |||
</li></ul> | |||
<table class="querytable" style="width:60%"> | |||
; | <tr> | ||
# | <td class="header"> Platform | ||
# | </td> | ||
# | <td class="header"> Releasetest: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-10-08&to=2012-10-10&target=16.0">report</a> | ||
</td> | |||
<td class="header"> Beta: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-10-08&to=2012-10-10&target=16.0">report</a> | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> <b>PASS</b> - 2010-10-08 | |||
</td> | |||
<td> <b>RESULTS</b> - yyyy-mm-dd | |||
</td></tr> | |||
<tr> | |||
<td> Windows Vista | |||
</td> | |||
<td> <b>PASS</b> - 2010-10-08 | |||
</td> | |||
<td> <b>RESULTS</b> - yyyy-mm-dd | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2010-10-08 | |||
</td> | |||
<td> <b>RESULTS</b> - yyyy-mm-dd | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2010-10-08 | |||
</td> | |||
<td> <b>RESULTS</b> - yyyy-mm-dd | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 32-bit | |||
</td> | |||
<td> <b>PASS</b>- see below - 2012-10-09 | |||
</td> | |||
<td> <b>RESULTS</b> - yyyy-mm-dd | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 64-bit | |||
</td> | |||
<td> <b>PASS</b>- see below - 2012-10-08 | |||
</td> | |||
<td> <b>RESULTS</b> - yyyy-mm-dd | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.6 | |||
</td> | |||
<td> <b>PASS</b> - 2010-10-08 | |||
</td> | |||
<td> <b>RESULTS</b> - yyyy-mm-dd | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.7 | |||
</td> | |||
<td> <b>PASS</b> - 2010-10-08 | |||
</td> | |||
<td> <b>RESULTS</b> - yyyy-mm-dd | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.8 | |||
</td> | |||
<td> <b>PASS</b> - 2010-10-08 | |||
</td> | |||
<td> <b>RESULTS</b> - yyyy-mm-dd | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2010-10-08 | |||
</td> | |||
<td> <b>RESULTS</b> - yyyy-mm-dd | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2010-10-08 | |||
</td> | |||
<td> <b>RESULTS</b> - yyyy-mm-dd | |||
</td></tr></table> | |||
<p><i>Windows 8: manual spotcheck until automation in place</i> | |||
</p> | |||
<ul><li> BETATEST | |||
<ul><li> <b>PASS</b> - Firefox 15.0.1 en-US de (partial) | |||
</li><li> <b>PASS</b> - Firefox 12.0 en-US fr (complete) | |||
</li></ul> | |||
</li><li> RELEASE | |||
<ul><li> <b>RESULT</b> - Firefox 15.0.1 en-US de (partial) | |||
</li><li> <b>RESULT</b> - Firefox 12.0 en-US de (complete) | |||
</li></ul> | |||
</li></ul> | |||
<dl><dt> Issues | |||
</dt></dl> | |||
<h3> Bug Triage RC </h3> | |||
<ol><li> <a href="http://bit.ly/RquQy9">VERIFYME (9)</a> : verify the fix and do some exploratory testing around the issue | |||
</li><li> <a href="http://bit.ly/Ppe9E0">FIXED (178)</a> : triage these bugs and test to ensure the fix is effective | |||
</li><li> <a href="http://bit.ly/PMaYVx">QAWANTED (8)</a> : triage these bugs and test to fulfill the QA request | |||
</li><li> <a href="http://bit.ly/OtEqj1">UNCONFIRMED (21)</a> : triage these bugs for potential new regressions | |||
</li></ol> | |||
<dl><dt> Results | |||
</dt></dl> | |||
<ul><li> <a href="http://bit.ly/MXYdqS">VERIFIED</a>: 89/304 | |||
</li><li> <a href="http://bit.ly/POyxuA">WONTVERIFY</a>: 32/304 | |||
</li><li> <a href="http://bit.ly/O09WSK">HELPWANTED</a>: 5/304 | |||
</li></ul> | |||
<h2> 16.0.1 </h2> | |||
<table class="querytable" style="width:30%"> | |||
<tr> | |||
<td style="width:10%"> | |||
</td> | |||
<td style="width:20%"> | |||
</td></tr> | |||
<tr> | |||
<td> <b>Builds</b> | |||
</td> | |||
<td> <a href="ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0.1-candidates/build1/">FTP</a>, <a href="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0.1-candidates/build1/">HTTP</a> | |||
</td></tr> | |||
<tr> | |||
<td> <b>ID</b> | |||
</td> | |||
<td> 20121010144125 | |||
</td></tr> | |||
<tr> | |||
<td> <b>Changeset</b> | |||
</td> | |||
<td> <a href="https://hg.mozilla.org/releases/mozilla-release/rev/7789ab68a99d">7789ab68a99d</a> (<a href="http://hg.mozilla.org/releases/mozilla-release/pushloghtml?fromchange=10fe550fadc6&tochange=7789ab68a99d">pushlog</a>) | |||
</td></tr></table> | |||
<dl><dt> Checklist | |||
</dt></dl> | |||
<ul><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Automated_Testing_7">Automated Testing</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Manual_Testing_7">Manual Testing</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Update_Testing_7">Update Testing</a> | |||
</li><li> <span class="fck_mw_template">{{ | }}</span> Final sign-off sent and updates qualified on Release channel | |||
</li></ul> | |||
<h3> Automated Testing 7 </h3> | |||
<p>The following are the detailed results of the automated Mozmill test runs. | |||
</p> | |||
<table class="querytable" style="width:50%"> | |||
; | <tr> | ||
<td class="header" style="width:20%"> Platform | |||
</td> | |||
<td class="header" style="width:30%"> RESULT: <a href="http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-10-10&to=2012-10-11">report</a> | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-10 | |||
</td></tr> | |||
<tr> | |||
<td> Windows Vista | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-10 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-10 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-10 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-10 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-10 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.6 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-10 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.7 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-10 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.8 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-10 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-10 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-10 | |||
</td></tr></table> | |||
<dl><dt> Test Failures (<a href="https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA">known bugs</a>) | |||
</dt></dl> | |||
<h3> Manual Testing 7 </h3> | |||
<p>There will be four bug verifications and testing around these bugs | |||
</p> | |||
<dl><dt> Results | |||
</dt></dl> | |||
<table class="querytable"> | |||
<tr> | |||
<td class="header"> Platform | |||
</td> | |||
<td class="header"> Tester | |||
</td> | |||
<td class="header"> 16.0.1 Candidates | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> juanb, ashughes, mwobensmith | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-10 | |||
</td></tr> | |||
<tr> | |||
<td> Windows Vista | |||
</td> | |||
<td> tester | |||
</td> | |||
<td> <b>RESULT</b> - 2012-10-10 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 32-bit | |||
</td> | |||
<td> tester | |||
</td> | |||
<td> <b>RESULT</b> - 2012-10-10 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 64-bit | |||
</td> | |||
<td> juanb, ashughes, mwobensmith | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-10 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.6 | |||
</td> | |||
<td> tester | |||
</td> | |||
<td> <b>RESULT</b> - 2012-10-10 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.7 | |||
</td> | |||
<td> juanb, ashughes, mwobensmith | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-10 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.8 | |||
</td> | |||
<td> tester | |||
</td> | |||
<td> <b>RESULT</b> - 2012-10-10 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 32-bit | |||
</td> | |||
<td> tester | |||
</td> | |||
<td> <b>RESULT</b> - 2012-10-10 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 64-bit | |||
</td> | |||
<td> juanb, ashughes, mwobensmith | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-10 | |||
</td></tr></table> | |||
<dl><dt> Issues Found | |||
</dt></dl> | |||
<h4> Breakpad End-to-End 7 </h4> | |||
<p>Test the end-to-end functionality in the crash reporting feature, across operating systems. | |||
</p> | |||
<ul><li> Use the <a href="http://ted.mielczarek.org/mozilla/crashme.html">Crash Me add-on</a> to trigger a crash | |||
</li><li> Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc | |||
</li><li> Repeatedly crash Firefox and submit your reports | |||
</li><li> Verify that your session restores properly | |||
</li><li> Verify your crash reports are submitted properly by going through about:crashes and clicking the reports | |||
</li><li> Try adding comments to your crash report and verify the crash-stats.com report shows the comment | |||
</li></ul> | |||
<h3> Update Testing 7 </h3> | |||
<ul><li> Partial: 13.0.1, 14.0.1, 15.0.1 | |||
</li><li> Complete: 12.0, 13.0 | |||
</li></ul> | |||
<table class="querytable" style="width:60%"> | |||
<tr> | |||
<td class="header"> Platform | |||
</td> | |||
<td class="header"> Releasetest: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-10-10&to=2012-10-10&target=16.0">report</a> | |||
</td> | |||
<td class="header"> Release: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-10-11&to=2012-10-11&target=16.0">report</a> | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-11 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-11 | |||
</td></tr> | |||
<tr> | |||
<td> Windows Vista | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-11 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-11 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-11 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-11 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-11 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-11 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 32-bit | |||
</td> | |||
<td> <b>Pass</b>- juanb - 2012-10-11 | |||
</td> | |||
<td> <b>Pass</b> - juanb - 2012-10-11 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 64-bit | |||
</td> | |||
<td> <b>Pass</b>- juanb - 2012-10-11 | |||
</td> | |||
<td> <b>Pass</b> - 2012-10-11 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.6 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-11 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-11 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.7 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-11 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-11 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.8 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-11 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-11 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-11 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-11 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-11 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-11 | |||
</td></tr></table> | |||
<p><i>Windows 8: manual spotcheck until automation in place</i> | |||
</p> | |||
<ul><li> RELEASETEST | |||
<ul><li> <b>PASS</b> - Firefox 16.0 en-US (partial) | |||
</li><li> <b>PASS</b> - Firefox 15.0.1 de (partial) | |||
</li></ul> | |||
</li><li> RELEASE | |||
<ul><li> <b>PASS</b> - Firefox 16.0 en-US (partial) | |||
</li><li> <b>PASS</b> - Firefox 12.0 fr (complete) | |||
</li></ul> | |||
</li></ul> | |||
<dl><dt> Issues | |||
</dt></dl> | |||
<h2> 16.0.2 </h2> | |||
<table class="querytable" style="width:30%"> | |||
<tr> | |||
<td style="width:10%"> | |||
</td> | |||
<td style="width:20%"> | |||
</td></tr> | |||
<tr> | |||
<td> <b>Builds</b> | |||
</td> | |||
<td> <a href="ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0.2-candidates/build1/">FTP</a>, <a href="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0.2-candidates/build1/">HTTP</a> | |||
</td></tr> | |||
<tr> | |||
<td> <b>ID</b> | |||
</td> | |||
<td> 20121024073032 | |||
</td></tr> | |||
<tr> | |||
<td> <b>Changeset</b> | |||
</td> | |||
<td> <a href="https://hg.mozilla.org/releases/mozilla-release/rev/3995ff235342">3995ff235342</a> (<a href="http://hg.mozilla.org/releases/mozilla-release/pushloghtml?fromchange=7789ab68a99d&tochange=3995ff235342">pushlog</a>) | |||
</td></tr></table> | |||
<dl><dt> Checklist | |||
</dt></dl> | |||
<ul><li> <span class="fck_mw_template">{{done|}}</span> <a href="Releases/Firefox 16/Test Plan#Automated_Testing_8">Automated Testing</a> | |||
</li><li> <span class="fck_mw_template">{{ok|}}</span> <a href="Releases/Firefox 16/Test Plan#Manual_Testing_8">Manual Testing</a> | |||
</li><li> <span class="fck_mw_template">{{ok|}}</span> <a href="Releases/Firefox 16/Test Plan#Update_Testing_8">Update Testing</a> | |||
</li><li> <span class="fck_mw_template">{{ | }}</span> Final sign-off sent and updates qualified on Beta channel | |||
</li></ul> | |||
<h3> Automated Testing 8 </h3> | |||
<p>The following are the detailed results of the automated Mozmill test runs. | |||
</p> | |||
<table class="querytable" style="width:50%"> | |||
<tr> | |||
<td class="header" style="width:20%"> Platform | |||
</td> | |||
<td class="header" style="width:30%"> RESULT: <a href="http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-10-24&to=2012-10-24">report</a> | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-24 | |||
</td></tr> | |||
<tr> | |||
<td> Windows Vista | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-24 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-24 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-24 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-24 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 8 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-24 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.6 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-24 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.7 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-24 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.8 | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-24 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 32-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-24 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 64-bit | |||
</td> | |||
<td> <b>PASS</b> - 2012-10-24 | |||
</td></tr></table> | |||
<dl><dt> Test Failures (<a href="https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA">known bugs</a>) | |||
</dt></dl> | |||
<h3> Manual Testing 8 </h3> | |||
<p>There will be X bug verifications and testing around these bugs | |||
</p> | |||
<dl><dt> Results | |||
</dt></dl> | |||
<table class="querytable"> | |||
<tr> | |||
<td class="header"> Platform | |||
</td> | |||
<td class="header"> Tester | |||
</td> | |||
<td class="header"> 16.0.2 Candidates | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> tester | |||
</td> | |||
<td> <b>RESULT</b> - 2012-10-24 | |||
</td></tr> | |||
<tr> | |||
<td> Windows Vista | |||
</td> | |||
<td> tester | |||
</td> | |||
<td> <b>RESULT</b> - 2012-10-24 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 32-bit | |||
</td> | |||
<td> tester | |||
</td> | |||
<td> <b>RESULT</b> - 2012-10-24 | |||
</td></tr> | |||
<tr> | |||
<td> Windows 7 64-bit | |||
</td> | |||
<td> tester | |||
</td> | |||
<td> <b>RESULT</b> - 2012-10-24 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.6 | |||
</td> | |||
<td> tester | |||
</td> | |||
<td> <b>RESULT</b> - 2012-10-24 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.7 | |||
</td> | |||
<td> Virgil | |||
</td> | |||
<td> <b>In progress</b> - 2012-10-24 | |||
</td></tr> | |||
<tr> | |||
<td> Mac OS X 10.8 | |||
</td> | |||
<td> tester | |||
</td> | |||
<td> <b>RESULT</b> - 2012-10-24 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 32-bit | |||
</td> | |||
<td> Ioana | |||
</td> | |||
<td> <b>In progress</b> - 2012-10-25 | |||
</td></tr> | |||
<tr> | |||
<td> Ubuntu 12.04 64-bit | |||
</td> | |||
<td> Ioana | |||
</td> | |||
<td> <b>In progress</b> - 2012-10-25 | |||
</td></tr></table> | |||
<dl><dt> Issues Found | |||
</dt></dl> | |||
<h4> Breakpad End-to-End 8 </h4> | |||
<p>Test the end-to-end functionality in the crash reporting feature, across operating systems. | |||
</p> | |||
<ul><li> Use the <a href="http://ted.mielczarek.org/mozilla/crashme.html">Crash Me add-on</a> to trigger a crash | |||
</li><li> Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc | |||
</li><li> Repeatedly crash Firefox and submit your reports | |||
</li><li> Verify that your session restores properly | |||
</li><li> Verify your crash reports are submitted properly by going through about:crashes and clicking the reports | |||
</li><li> Try adding comments to your crash report and verify the crash-stats.com report shows the comment | |||
</li></ul> | |||
<h3> Update Testing 8 </h3> | |||
<ul><li> Partial: 14.0.1, 15.0.1, 16.0.1 | |||
</li><li> Complete: 12.0, 13.0 | |||
</li></ul> | |||
<table class="querytable" style="width:60%"> | |||
<tr> | |||
<td class="header"> Platform | |||
</td> | |||
<td class="header"> Releasetest: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-10-24&to=2012-10-25&target=16.0">report</a> | |||
</td> | |||
<td class="header"> Beta: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-10-24&to=2012-10-25&target=16.0">report</a> | |||
</td></tr> | |||
<tr> | |||
<td> Windows XP | |||
</td> | |||
<td> <b>RESULT</b> - 2012-10-24 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-10-24 | |||
</td></tr> | |||
<tr> | |||
<td> Windows Vista | |||
</td> | |||
<td> <b>RESULT</b> - 2012-10-24 | |||
</td> | |||
<td> <b>RESULT</b> - 2012-10- | |||
Revision as of 06:31, 25 October 2012
<a href="Releases/Firefox 15/Test Plan">« Firefox 15</a> | <a href="Releases/Firefox_16/Test_Plan" class="selflink">Firefox 16</a> | <a href="Releases/Firefox 17/Test Plan">Firefox 17 »</a>
Firefox 16 Test Plan
Summary
Milestone | <a href="https://wiki.mozilla.org/Firefox/Flight_Tracking#Firefox_16">Firefox 16</a> |
Status | Currently in Beta -- merging to Release on October 8, 2012 |
Lead | <a href="mailto:jbecerra@mozilla.com">Juan Becerra</a> (irc: juanb) |
2nd | <a href="mailto:ahughes@mozilla.com">Anthony Hughes</a> (irc: ashughes) |
Builds | <a href="https://www.mozilla.org/beta/">beta.mozilla.org</a> |
The following is the test plan for Firefox 16 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 16. After the release this document will be used as an archive of what was done to validate this release.
If you have some free time, please pick a task below and get in touch with one of the leads.
Schedule
The following is a description of the tasks which need to be completed throughout this release.
- 2012-06-04: [DONE] Nightly Merge
- 2012-07-17: [DONE] Aurora Merge
- 2012-08-28: [DONE] Pre-Beta feature sign-offs
- 2012-08-30: [DONE] Beta 1
- 2012-09-06: [DONE] Beta 2
- 2012-09-13: [ON TRACK] Beta 3
- 2012-09-20: {{|}} Beta 4
- 2012-09-27: {{|}} Beta 5
- 2012-10-04: {{|}} Beta 6
- 2012-10-06: {{|}} Release Candidate
- 2012-10-09: {{|}} Final
Nightly
Merge
- Functional Automation - result
- Update Spotchecks - result
Features
See the <a href="https://wiki.mozilla.org/Features/Release_Tracking">Feature Release Tracking</a> wiki for a general overview
Feature | Owner | Status | Tests |
<a href="Platform/Features/Incremental GC">Incremental GC</a> | Ioana Budnar | On track | |
<a href="Opt-in activation for plugins">Opt-in activation for plugins</a> | Paul Silaghi | On track | |
<a href="DevTools/Features/DeveloperToolbar">Developer Toolbar</a> | Virgil Dicu | On track | |
<a href="DevTools/Features/GCLI">Command Line</a> | Mihaela Velimiroviciu | On track | |
<a href="Show PDF inline">Show PDF inline</a> | Mihaela Velimiroviciu | Disabled until Fx18(bug 786012) | |
<a href="User:P.A./Panel-based Download Manager">Panel-based Download Manager</a> | Simona Badau | Delayed to Fx18 | |
<a href="OS X 10.7 support">OS X 10.7 support</a> | Vlad Ghetiu | Not implemented |
Endurance Automation
The purpose of this section is to track any potential memory regressions via spikes reported in the dashboard.
- What to Do
- Check the <a href="http://mozmill-ci.blargon7.com/#/endurance/charts?platform=All&from=2012-06-05&to=2012-07-17">Endurance Charts</a>
- If you see an anomalous event (ie. spike or dip), get the report for the specific event
- If the event happens twice in a row, file a bug under Mozmill Tests
- Notify <a href="mailto:ahughes@mozilla.com">Anthony Hughes</a> of the issue
- Record the event in this wiki below
- Current Status (Mihaela Velimiroviciu) - Last Updated Jul 18th, 2012
ID | Summary | Status | Priority |
---|---|---|---|
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=773632">773632</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=773632">High results on mozmill endurance test on Mac 10.7.4_x64 since Jul 5</a> | NEW | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=764823">764823</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=764823">Memory spike discovered by endurance tests on Nightly Linux and Mac between June 12 and 13</a> | RESOLVED | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=767348">767348</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=767348">Memory consumption shows an ascending trend on Nightly/Win 6.1 x64</a> | RESOLVED | -- |
Functional Automation
The purpose of this section is to track any potential functional regressions via results reported in the dashboard.
- What to Do
- Check the <a href="http://mozmill-release.blargon7.com/#/functional/reports?branch=13.0&platform=All&from=2012-06-04&to=2012-07-17">testrun reports</a>
- If you see any failing tests, check that there are not any <a href="http://bit.ly/wZL8zF">bugs on file</a> for it already
- If not, file a bug under Mozmill Tests
- Notify <a href="mailto:ahughes@mozilla.com">Anthony Hughes</a> of the issue
- Record the event in this wiki below
- Current Status (Vlad Maniac / Alex Lakatos) - Last Updated
Bug Triage
The purpose of this section is to triage UNCONFIRMED incoming bugs.
- What to Do
- Check the <a href="http://bit.ly/HXTYdN">UNCONFIRMED</a> bugs
- Test the bugs in the latest Firefox Nightly
- If you can't reproduce the bug, ask for the reporter to provide more information
- If you reproduce the bug, add your testing information and mark the bug NEW
- Notify <a href="mailto:ahughes@mozilla.com">Anthony Hughes</a> of the NEW issue
- Current Status (Vlad Ghetiu / Virgil Dicu) - Last Updated
<a href="http://bit.ly/z2MmKE">Result Spreadsheet</a>
Aurora Merge
The following are the detailed results of the automated BFT test runs.
Platform | Result: <a href="http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-07-18&to=2012-07-20">report</a> |
Windows 2000 | PASS - 2012-07-19 |
Windows XP | PASS - 2012-07-19 |
Windows Vista | PASS - 2012-07-19 |
Windows 7 32-bit | PASS - 2012-07-19 |
Windows 7 64-bit | PASS - 2012-07-19 |
Mac OS X 10.6 | PASS - 2012-07-19 |
Mac OS X 10.7 | PASS - 2012-07-19 |
Ubuntu 11.04 32-bit | PASS - 2012-07-19 |
Ubuntu 11.04 64-bit | PASS - 2012-07-19 |
- Known Issues
ID | Summary | Status | Priority |
---|---|---|---|
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=770117">770117</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=770117">Test failure "Number of visible rows should equal max rows - '1' should equal '6'" in /testAwesomeBar/testVisibleItemsMax.js</a> | RESOLVED | P2 |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=719982">719982</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=719982">Failure in testAddons_RestartlessExtensionWorksAfterRestart | Modal dialog has been found and processed</a> | RESOLVED | P2 |
Updates
The following are details of update verifications for Firefox 15 merge to Aurora
- Auroratest
- {{|}} [tester] Windows XP:
- {{|}} [tester] Windows 7:
- {{|}} [tester] Mac OS X 10.6:
- {{|}} [tester] Ubuntu 11.04:
- Aurora
- {{|}} [tester] Windows XP:
- {{|}} [tester] Windows 7:
- {{|}} [tester] Mac OS X 10.6:
- {{|}} [tester] Ubuntu 11.04:
- Issues Found
Manual Spot Checks and Feature Sign-Off
Are all features implemented at this point working?
Manual spot Checks [spreadsheet]
- Platforms: Windows XP, Windows 7 x64, Ubuntu 11.10, Mac 10.6
- Found known issues:
- New issues:
- Status:
Run the Firefox 16 Aurora Merge Feature Sign-Off tests in <a href="https://moztrap.mozilla.org/runtests/">MozTrap</a> and report results below.
Feature | Owner | Win32 | Mac | Linux |
<a href="https://wiki.mozilla.org/User:P.A./Panel-based_Download_Manager">Panel Based Download Manager</a> | Simona Badau | PASS - 2012-07-20 | PASS - 2012-07-19 | PASS - 2012-07-20 |
<a href="https://wiki.mozilla.org/OS_X_10.7_support">OS X 10.7 support</a> - Not landed | Vlad Ghetiu | N/A - 2012-07-19 | N/A - 2012-07-19 | N/A - 2012-07-19 |
<a href="https://wiki.mozilla.org/Opt-in_activation_for_plugins">Opt-in activation for plugins</a> | Paul Silaghi | PASS - 2012-07-20 | PASS - 2012-07-20 | PASS - 2012-07-20 |
<a href="https://wiki.mozilla.org/Speedy_Session_Restore">Speedy Session Restore</a> | Paul Silaghi | PASS - 2012-07-19 | PASS - 2012-07-19 | PASS - 2012-07-19 |
<a href="https://wiki.mozilla.org/DevTools/Features/DeveloperToolbar">Developer Toolbar</a> | Virgil Dicu | PASS - 2012-07-19 | PASS - 2012-07-19 | PASS - 2012-07-19 |
<a href="https://wiki.mozilla.org/DevTools/Features/GCLI">Command Line</a> | Mihaela Velimiroviciu | PASS - 2012-07-20 | PASS - 2012-07-20 | PASS - 2012-07-20 |
<a href="https://wiki.mozilla.org/Show_PDF_inline">Show PDF Inline</a> | Mihaela Velimiroviciu | PASS - 2012-07-20 | PASS - 2012-07-20 | PASS - 2012-07-20 |
<a href="Platform/Features/Incremental GC/TestPlan">
Incremental Garbage Collection</a> |
Ioana Budnar | PASS - 2012-07-19 | PASS - 2012-07-20 | PASS - 2012-07-20 |
Notes:
- OS X 10.7 support - not landed
Smoke tests
Run the smoketests in MozTrap and report results below.
Platform | Assignee | Result: <a href="https://moztrap.mozilla.org/results/cases/?filter-run=62">report</a> |
Windows XP | Simona | PASS - 2012-07-19 |
Windows 7 64-bit | Paul | PASS - 2012-07-20 |
Mac OS X 10.7.4 | VladG | PASS - 2012-07-19 |
Ubuntu 12.04 32-bit | Virgil | PASS - 2012-07-19 |
Known Issues
<thead> </thead> <tbody> </tbody>ID | Summary | Status | Priority |
---|---|---|---|
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=692446">692446</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=692446">Firefox prompts to install windows media player plugin even though it is already installed</a> | NEW | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=775491">775491</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=775491">HTML5 unresponsive script errors on YouTube (unresponsive script name: html5player-vflGWJljH.js)</a> | RESOLVED | -- |
Pre-beta Feature testing and Sign-off
Manual spot Checks [spreadsheet]
- Platforms: Windows XP, Windows 7 x64, Ubuntu 11.10, Mac 10.6
- Found known issues:
- New issues:
- Status:
Run the Firefox 16 Aurora Merge Feature Sign-Off tests in <a href="https://moztrap.mozilla.org/runtests/">MozTrap</a> and report results below.
Feature | Owner | Win32 | Mac | Linux |
<a href="https://wiki.mozilla.org/In-content_preferences">In-content_preferences</a> | Mihaela Velimiroviciu | disabled - 2012-08-24 | disabled - 2012-08-24 | disabled - 2012-08-24 |
<a href="https://wiki.mozilla.org/User:P.A./Panel-based_Download_Manager">Panel Based Download Manager</a> | Simona Badau | disabled - 2012-08-24 | disabled - 2012-08-24 | disabled - 2012-08-24 |
<a href="https://wiki.mozilla.org/OS_X_10.7_support">OS X 10.7 support</a> | Vlad Ghetiu | NA - 2012-08-24 | NA - 2012-08-24 | NA - 2012-08-24 |
<a href="https://wiki.mozilla.org/Opt-in_activation_for_plugins">Opt-in activation for plugins</a> | Paul Silaghi | disabled - 2012-08-24 | disabled - 2012-08-24 | disabled - 2012-08-24 |
<a href="https://wiki.mozilla.org/Speedy_Session_Restore">Speedy Session Restore</a> | Paul Silaghi | NA - 2012-08-24 | NA - 2012-08-24 | NA - 2012-08-24 |
<a href="https://wiki.mozilla.org/DevTools/Features/DeveloperToolbar">Developer Toolbar</a> | Virgil Dicu | PASS - 2012-08-24 | PASS - 2012-08-24 | PASS - 2012-08-24 |
<a href="https://wiki.mozilla.org/DevTools/Features/GCLI">Command Line</a> | Mihaela Velimiroviciu | PASS - 2012-08-24 | PASS - 2012-08-24 | PASS - 2012-08-24 |
<a href="https://wiki.mozilla.org/Show_PDF_inline">Show PDF Inline</a> | Mihaela Velimiroviciu | PASS - 2012-08-24 | PASS - 2012-08-24 | PASS - 2012-08-24 |
<a href="Platform/Features/Incremental GC/TestPlan">
Incremental Garbage Collection</a> |
Ioana Budnar | PASS - 2012-08-24 | PASS - 2012-08-24 | PASS - 2012-08-24 |
Notes:
- OS X 10.7 support: Not yet implemented.
- Speedy Session Restore: backed out.
Smoke tests
Run the smoketests in MozTrap and report results below.
Platform | Assignee | Result: <a href="https://moztrap.mozilla.org/results/cases/?filter-run=62">report</a> |
Windows XP | Simona | PASS - 2012-08-24 |
Windows 7 64-bit | VladG | PASS - 2012-08-24 |
Mac OS X 10.7.4 | VladG | PASS - 2012-08-24 |
Ubuntu 12.04 32-bit | Virgil | PASS - 2012-08-24 |
Known Issues
<thead> </thead> <tbody> </tbody>ID | Summary | Status | Priority |
---|---|---|---|
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=775082">775082</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=775082">Video controls disappear after dragging the tabs</a> | NEW | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=784289">784289</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=784289">Inspector button remains open in devtoolbar when closing the Inspector via x button</a> | RESOLVED | -- |
Beta 1
Builds | <a href="ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b1-candidates/build1/">FTP</a>, <a href="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b1-candidates/build1/">HTTP</a> |
ID | 20120828083259 |
Changeset | <a href="http://hg.mozilla.org/releases/mozilla-beta/rev/c4e1ca772267">c4e1ca772267</a> (<a href="http://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=2099c2bea208&tochange=c4e1ca772267">pushlog</a>) |
- Checklist
- [DONE] <a href="Releases/Firefox 16/Test Plan#Automated_Testing">Automated Testing</a>
- [DONE] <a href="Releases/Firefox 16/Test Plan#Manual_Testing">Manual Testing</a>
- [DONE] <a href="Releases/Firefox 16/Test Plan#Update_Testing">Update Testing</a>
- [DONE] <a href="Releases/Firefox 16/Test Plan#Bug_Triage">Bug Triage</a>
- [DONE] Peer Sign-off
- {{ |}} Final sign-off sent and updates qualified on Beta channel
Automated Testing
The following are the detailed results of the automated Mozmill test runs.
Platform | PASS: <a href="http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-08-28&to=2012-08-28">report</a> |
Windows XP | PASS - 2012-08-28 |
Windows Vista | PASS - 2012-08-28 |
Windows 7 32-bit | PASS - 2012-08-28 |
Windows 7 64-bit | PASS - 2012-08-28 |
Windows 8 32-bit | N/A (pending IT) |
Windows 8 64-bit | N/A (pending IT) |
Mac OS X 10.6 | PASS - 2012-08-28 |
Mac OS X 10.7 | PASS - 2012-08-28 |
Mac OS X 10.8 | PASS - 2012-08-28 |
Ubuntu 12.04 32-bit | PASS - 2012-08-28 |
Ubuntu 12.04 64-bit | PASS - 2012-08-28 |
- Test Failures (<a href="https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA">known bugs</a>)
Manual Testing
Operating System | Tester | Plug-in Compatibility | Add-on Compatibility | Web Compatibility |
Windows XP | Simona | PASS - 2012-08-29 | PASS - 2012-08-29 | PASS - 2012-08-29 |
Windows 7 | Mihaela | PASS - 2012-08-29 | PASS - 2012-08-29 | PASS - 2012-08-29 |
Windows 8 | Paul | PASS - 2012-08-29 | PASS - 2012-08-29 | PASS - 2012-08-29 |
Mac OS X | VladG | PASS - 2012-08-29 | PASS - 2012-08-29 | PASS - 2012-08-29 |
Ubuntu 12.04 | Virgil | PASS - 2012-08-29 | PASS - 2012-08-29 | PASS - 2012-08-29 |
Operating System | Tester | Web Mail | Mac 10.8 Signing | Breakpad |
Windows XP | Simona | PASS - 2012-08-29 | N/A | PASS - 2012-08-29 |
Windows 7 | Mihaela | PASS - 2012-08-29 | N/A | PASS - 2012-08-29 |
Windows 8 | Paul | PASS - 2012-08-29 | N/A | PASS - 2012-08-29 |
Mac OS X | VladG | PASS - 2012-08-29 | PASS - 2012-08-29 | PASS - 2012-08-29 |
Ubuntu 12.04 | Virgil | PASS - 2012-08-29 | N/A | PASS - 2012-08-29 |
Plug-in Compatibility
Install the following plug-ins and go to websites which use them, looking for rendering issues, functional issues, performance issues, and crashes.
- Adobe Flash Player
- <a href="http://get.adobe.com/flashplayer/">11.3 Release</a>
- <a href="http://labs.adobe.com/downloads/flashplayer11-4.html">Flash 11.4 beta</a> (make sure you can autoupdate)
- <a href="http://www.microsoft.com/silverlight/">Microsoft Silverlight</a>
- <a href="http://www.real.com/">RealPlayer</a>
- Real Player plugin crashed when opening pages with content managed by the plugin: <a href="http://www.slp3d2.com/launchsiteglobal/test.cfm">click</a>, <a href="http://cf.unc.edu/skillstest/take/realplayer.htm">click</a> - bug 667399
- <a href="http://www.apple.com/quicktime/download/">QuickTime</a>
Add-on Compatibility
Test the following websites for rendering issues, functional issues, performance issues, and crashes. Pay particular attention to any problems seen after updating Firefox with add-ons installed.
- <a href="https://addons.mozilla.org/en-US/firefox/compatibility/15.0">Top AMO add-ons</a>
- AVG toolbar (installed via AVG software)
- AVG Safe Search 12.0.0.1912 is not compatible with Firefox 15 beta.
- AVG Security Toolbar can't be installed on Win XP - we get a blue screen while installing it.
- Norton toolbar (installed via Norton software)
- Not currently available. Investigating if we can get this software.
- <a href="http://sg.toolbar.yahoo.com/">Yahoo toolbar</a>
- <a href="http://bar.yandex.ru/en/">Yandex toolbar</a>
- <a href="http://babylon-toolbar.software.informer.com/download/">Babylon toolbar</a>
- <a href="http://lavasoft.com/">Ad-Aware Antivirus</a>
- The toolbar can't be installed on Firefox.
Web Compatibility
Test the following websites for rendering issues, functional issues, performance issues, and crashes
- <a href="http://www.alexa.com/topsites">Overall Top 10</a>
- <a href="http://www.alexa.com/topsites/countries">Top by Country</a> (pick a random site from three random countries)
- <a href="http://www.alexa.com/topsites/category">Top by Category</a> (pick a random site from three random categories)
- Webmail clients like gmail, hotmail, and yahoo mail (pay particular attention to any built in web messenger software)
- Try Google's <a href="http://support.google.com/maps/bin/answer.py?hl=en&answer=1630790">MapsGL demo</a>
- Try YouTube's <a href="http://www.youtube.com/html5">HTML5 demo</a>
Web Mail
Test the following websites for rendering issues, functional issues, performance issues, and crashes. In particular make sure email, attachments, web messenger, and video chat are working as expected.
- <a href="https://mail.google.com/">Google Mail</a>
- <a href="https://mail.yahoo.com/">Yahoo Mail</a>
- <a href="https://www.hotmail.com">Hotmail</a>
- <a href="https://www.outlook.com/">Outlook.com</a>
Mac 10.8 Signing
NOTE: Test needs to be run from a clean Mac OSX 10.8 install. Please keep a vanilla image handy so you can restore once completing this test (<a href="https://people.mozilla.com/~stmichaud/Gatekeeper/MLGatekeeper.html">more info</a>).
- Download the .dmg file and drag the Firefox.app file to a location on your computer
- Open a terminal window and cd to the folder Firefox.app is located
- Run codesign -vv Firefox.app in terminal, output should contain valid on disk and satisfies its Designated Requirement
- Run xattr Firefox.app in terminal, output should contain com.apple.quarantine
- Double click the Firefox.app icon, you should see a warning about the program being provided by an unknown developer or having been downloaded from the Internet
- Click "open" and browse to any website, Firefox should start and you should be able to browse
- Quit Firefox and double click the Firefox.app icon again, you should not be reprompted for the warning in step 5, firefox should start up and browse without any issues
Spot check build after this test
- Try all the menu item functions
- Gestures
- Plug-in compatibility
- Top-site compatibility
- Add-on compatibility
- Software update
Breakpad End-to-End
Test the end-to-end functionality in the crash reporting feature, across operating systems.
- Use the <a href="http://ted.mielczarek.org/mozilla/crashme.html">Crash Me add-on</a> to trigger a crash
- Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc
- Repeatedly crash Firefox and submit your reports
- Verify that your session restores properly
- Verify your crash reports are submitted properly by going through about:crashes and clicking the reports
- Try adding comments to your crash report and verify the crash-stats.com report shows the comment
Update Testing
- Partial: 15.0b6
- Complete: 15.0b5 and earlier
Platform | Releasetest: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-08-28&to=2012-08-30&target=16.0">report</a> | Beta: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-08-28&to=2012-08-30&target=16.0">report</a> |
Windows XP | PASS - 2012-08-29 | PASS - 2012-08-30 |
Windows Vista | PASS - 2012-08-29 | PASS - 2012-08-30 |
Windows 7 32-bit | PASS - 2012-08-29 | PASS - 2012-08-30 |
Windows 7 64-bit | PASS - 2012-08-29 | PASS - 2012-08-30 |
Windows 8 32-bit | manual - see below | manual - see below |
Windows 8 64-bit | manual - see below | manual - see below |
Mac OS X 10.6 | PASS - 2012-08-29 | PASS - 2012-08-30 |
Mac OS X 10.7 | PASS - 2012-08-29 | PASS - 2012-08-30 |
Mac OS X 10.8 | PASS - 2012-08-29 | PASS - 2012-08-30 |
Ubuntu 12.04 32-bit | PASS - 2012-08-29 | PASS - 2012-08-30 |
Ubuntu 12.04 64-bit | PASS - 2012-08-29 | PASS - 2012-08-30 |
Windows 8: manual spotcheck until automation in place
- RELEASETEST
- PASS - Firefox 15.0b6 en-US (partial)
- PASS - Firefox 15.0b6 de (partial)
- PASS - Firefox 14.0b6 en-US (complete)
- PASS - Firefox 14.0b6 zh-TW (complete)
- BETA
- PASS - Firefox 15.0b6 en-US (partial)
- PASS - Firefox 15.0b6 de (partial)
- PASS - Firefox 14.0b6 en-US (complete)
- PASS - Firefox 14.0b6 zh-TW (complete)
- Issues
Bug Triage
- <a href="http://bit.ly/RquQy9">VERIFYME</a> (3): verify the fix and do some exploratory testing around the issue
- <a href="http://bit.ly/Ppe9E0">FIXED</a> (221): triage these bugs and test to ensure the fix is effective
- <a href="http://bit.ly/PMaYVx">QAWANTED</a> (3): triage these bugs and test to fulfill the QA request
- <a href="http://bit.ly/OtEqj1">UNCONFIRMED</a> (3): triage these bugs for potential new regressions
- Results
- <a href="http://bit.ly/MXYdqS">VERIFIED</a>: 50/221
- <a href="http://bit.ly/POyxuA">WONTVERIFY</a>: 21/221
- <a href="http://bit.ly/O09WSK">HELPWANTED</a>: 14/221
Beta 2
Builds | <a href="ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b2-candidates/build1/">FTP</a>, <a href="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b2-candidates/build1/">HTTP</a> |
ID | 20120904124322 |
Changeset | <a href="http://hg.mozilla.org/releases/mozilla-beta/rev/cad589192f3c">cad589192f3c</a> (<a href="http://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=c4e1ca772267&tochange=cad589192f3c">pushlog</a>) |
- Checklist
- [DONE] <a href="Releases/Firefox 16/Test Plan#Automated_Testing_2">Automated Testing</a>
- [DONE] <a href="Releases/Firefox 16/Test Plan#Manual_Testing_2">Manual Testing</a>
- [DONE] <a href="Releases/Firefox 16/Test Plan#Update_Testing_2">Update Testing</a>
- [DONE] <a href="Releases/Firefox 16/Test Plan#Bug_Triage_2">Bug Triage</a>
- [DONE] Peer Sign-off
- {{ |}} Final sign-off sent and updates qualified on Beta channel
Automated Testing 2
The following are the detailed results of the automated Mozmill test runs.
Platform | PASS: <a href="http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-09-04&to=2012-09-05">report</a> |
Windows XP | PASS - 2012-09-05 |
Windows Vista | PASS - 2012-09-05 |
Windows 7 32-bit | PASS - 2012-09-05 |
Windows 7 64-bit | PASS - 2012-09-05 |
Windows 8 32-bit | N/A (pending IT) |
Windows 8 64-bit | N/A (pending IT) |
Mac OS X 10.6 | PASS - 2012-09-05 |
Mac OS X 10.7 | PASS - 2012-09-05 |
Mac OS X 10.8 | PASS - 2012-09-05 |
Ubuntu 12.04 32-bit | PASS - 2012-09-05 |
Ubuntu 12.04 64-bit | PASS - 2012-09-05 |
- Test Failures (<a href="https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA">known bugs</a>)
Manual Testing 2
Operating System | Tester | Plug-in Compatibility | Add-on Compatibility | Web Compatibility |
Windows XP | Ioana | PASS - 2012-09-07 | PASS - 2012-09-07 | PASS - 2012-09-07 |
Windows 7 | Mihaela | PASS - 2012-09-07 | PASS - 2012-09-07 | PASS - 2012-09-07 |
Windows 8 | Virgil | PASS - 2012-09-07 | PASS - 2012-09-07 | PASS - 2012-09-07 |
Mac OS X | VladG | PASS - 2012-09-07 | PASS - 2012-09-07 | PASS - 2012-09-07 |
Ubuntu 12.04 | Ioana | PASS - 2012-09-07 | PASS - 2012-09-07 | PASS - 2012-09-07 |
Operating System | Tester | Web Mail | Mac 10.8 Signing | Breakpad |
Windows XP | Virgil | PASS - 2012-09-07 | N/A | PASS - 2012-09-07 |
Windows 7 | Mihaela | PASS - 2012-09-07 | N/A | PASS - 2012-09-07 |
Windows 8 | Virgil | PASS - 2012-09-07 | N/A | PASS - 2012-09-07 |
Mac OS X | VladG | PASS - 2012-09-07 | PASS - 2012-09-07 | PASS - 2012-09-07 |
Ubuntu 12.04 | Ioana | PASS - 2012-09-07 | N/A | PASS - 2012-09-07 |
Plug-in Compatibility 2
Install the following plug-ins and go to websites which use them, looking for rendering issues, functional issues, performance issues, and crashes.
- Adobe Flash Player
- <a href="http://get.adobe.com/flashplayer/">11.4 Release</a>
- <a href="http://www.microsoft.com/silverlight/">Microsoft Silverlight</a>
- <a href="http://www.real.com/">RealPlayer</a>
- Real Player plugin crashed when opening pages with content managed by the plugin: <a href="http://www.slp3d2.com/launchsiteglobal/test.cfm">click</a>, <a href="http://cf.unc.edu/skillstest/take/realplayer.htm">click</a> - bug 667399
- <a href="http://www.apple.com/quicktime/download/">QuickTime</a>
Add-on Compatibility 2
Test the following websites for rendering issues, functional issues, performance issues, and crashes. Pay particular attention to any problems seen after updating Firefox with add-ons installed.
- <a href="https://addons.mozilla.org/en-US/firefox/compatibility/15.0">Top AMO add-ons</a>
- AVG toolbar (installed via AVG software)
- Norton toolbar (installed via Norton software)
- <a href="http://sg.toolbar.yahoo.com/">Yahoo toolbar</a>
- <a href="http://bar.yandex.ru/en/">Yandex toolbar</a>
- <a href="http://babylon-toolbar.software.informer.com/download/">Babylon toolbar</a>
- <a href="http://lavasoft.com/">Ad-Aware Antivirus</a>
Web Compatibility 2
Test the following websites for rendering issues, functional issues, performance issues, and crashes
- <a href="http://www.alexa.com/topsites">Overall Top 10</a>
- <a href="http://www.alexa.com/topsites/countries">Top by Country</a> (pick a random site from three random countries)
- <a href="http://www.alexa.com/topsites/category">Top by Category</a> (pick a random site from three random categories)
- Webmail clients like gmail, hotmail, and yahoo mail (pay particular attention to any built in web messenger software)
- Try Google's <a href="http://support.google.com/maps/bin/answer.py?hl=en&answer=1630790">MapsGL demo</a>
- Try YouTube's <a href="http://www.youtube.com/html5">HTML5 demo</a>
Web Mail 2
Test the following websites for rendering issues, functional issues, performance issues, and crashes. In particular make sure email, attachments, web messenger, and video chat are working as expected.
- <a href="https://mail.google.com/">Google Mail</a>
- <a href="https://mail.yahoo.com/">Yahoo Mail</a>
- <a href="https://www.hotmail.com">Hotmail</a>
- <a href="https://www.outlook.com/">Outlook.com</a>
Mac 10.8 Signing 2
NOTE: Test needs to be run from a clean Mac OSX 10.8 install. Please keep a vanilla image handy so you can restore once completing this test (<a href="https://people.mozilla.com/~stmichaud/Gatekeeper/MLGatekeeper.html">more info</a>).
- Download the .dmg file and drag the Firefox.app file to a location on your computer
- Open a terminal window and cd to the folder Firefox.app is located
- Run codesign -vv Firefox.app in terminal, output should contain valid on disk and satisfies its Designated Requirement
- Run xattr Firefox.app in terminal, output should contain com.apple.quarantine
- Double click the Firefox.app icon, you should see a warning about the program being provided by an unknown developer or having been downloaded from the Internet
- Click "open" and browse to any website, Firefox should start and you should be able to browse
- Quit Firefox and double click the Firefox.app icon again, you should not be reprompted for the warning in step 5, firefox should start up and browse without any issues
Spot check build after this test
- Try all the menu item functions
- Gestures
- Plug-in compatibility
- Top-site compatibility
- Add-on compatibility
- Software update
Breakpad End-to-End 2
Test the end-to-end functionality in the crash reporting feature, across operating systems.
- Use the <a href="http://ted.mielczarek.org/mozilla/crashme.html">Crash Me add-on</a> to trigger a crash
- Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc
- Repeatedly crash Firefox and submit your reports
- Verify that your session restores properly
- Verify your crash reports are submitted properly by going through about:crashes and clicking the reports
- Try adding comments to your crash report and verify the crash-stats.com report shows the comment
Update Testing 2
- Partial: 16.0b1
- Complete: 15.0b6 and earlier
Platform | Releasetest: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-09-05&to=2012-09-06&target=16.0">report</a> | Beta: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-09-06&to=2012-09-07&target=16.0">report</a> |
Windows XP | PASS - 2012-09-06 | PASS - 2012-09-06 |
Windows Vista | PASS - 2012-09-06 | PASS - 2012-09-06 |
Windows 7 32-bit | PASS - 2012-09-06 | PASS - 2012-09-06 |
Windows 7 64-bit | PASS - 2012-09-06 | PASS - 2012-09-06 |
Windows 8 32-bit | manual - see below | manual - see below |
Windows 8 64-bit | PASS - see below | PASS - see below |
Mac OS X 10.6 | PASS - 2012-09-06 | PASS - 2012-09-06 |
Mac OS X 10.7 | PASS - 2012-09-06 | PASS - 2012-09-06 |
Mac OS X 10.8 | PASS - 2012-09-06 | PASS - 2012-09-06 |
Ubuntu 12.04 32-bit | PASS - 2012-09-06 | PASS - 2012-09-06 |
Ubuntu 12.04 64-bit | PASS - 2012-09-06 | PASS - 2012-09-06 |
Windows 8: manual spotcheck until automation in place
- RELEASETEST
- PASS - Firefox 16.0b1 en-US (partial)
- PASS - Firefox 15.0b6 de (complete)
- BETA
- PASS - Firefox 16.0b1 en-US (partial)
- PASS - Firefox 15.0b6 de (complete)
- Issues
Bug Triage 2
- <a href="http://bit.ly/RquQy9">VERIFYME</a> : verify the fix and do some exploratory testing around the issue
- <a href="http://bit.ly/Ppe9E0">FIXED</a> : triage these bugs and test to ensure the fix is effective
- <a href="http://bit.ly/PMaYVx">QAWANTED</a> : triage these bugs and test to fulfill the QA request
- <a href="http://bit.ly/OtEqj1">UNCONFIRMED</a> : triage these bugs for potential new regressions
- Results
- <a href="http://bit.ly/MXYdqS">VERIFIED</a>: Verified/Fixed Ratio
- <a href="http://bit.ly/POyxuA">WONTVERIFY</a>: Won't Verify/Fixed Ratio
- <a href="http://bit.ly/O09WSK">HELPWANTED</a>: Help Wanted/Fixed Ratio
Beta 3
Builds | <a href="ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b3-candidates/build1/">FTP</a>, <a href="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b3-candidates/build1/">HTTP</a> |
ID | 20120911232325 |
Changeset | <a href="http://hg.mozilla.org/releases/mozilla-beta/rev/de3f06a549de">de3f06a549de</a> (<a href="http://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=cad589192f3c&tochange=de3f06a549de">pushlog</a>) |
- Checklist
- [DONE] <a href="Releases/Firefox 16/Test Plan#Automated_Testing_2">Automated Testing</a>
- [DONE] <a href="Releases/Firefox 16/Test Plan#Manual_Testing_2">Manual Testing</a>
- [DONE] <a href="Releases/Firefox 16/Test Plan#Update_Testing_2">Update Testing</a>
- [ON TRACK] <a href="Releases/Firefox 16/Test Plan#Bug_Triage_2">Bug Triage</a>
- [DONE] Peer Sign-off
- [DONE] Final sign-off sent and updates qualified on Beta channel
Automated Testing 3
The following are the detailed results of the automated Mozmill test runs.
Platform | PASS: <a href="http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-09-11&to=2012-09-12">report</a> |
Windows XP | PASS - 2012-09-12 |
Windows Vista | PASS - 2012-09-12 |
Windows 7 32-bit | PASS - 2012-09-12 |
Windows 7 64-bit | PASS - 2012-09-12 |
Windows 8 32-bit | PASS - 2012-09-12 |
Windows 8 64-bit | PASS - 2012-09-12 |
Mac OS X 10.6 | PASS - 2012-09-12 |
Mac OS X 10.7 | PASS - 2012-09-12 |
Mac OS X 10.8 | PASS - 2012-09-12 |
Ubuntu 12.04 32-bit | PASS - 2012-09-12 |
Ubuntu 12.04 64-bit | PASS - 2012-09-12 |
- Test Failures (<a href="https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA">known bugs</a>)
Manual Testing 3
Operating System | Tester | Plug-in Compatibility | Add-on Compatibility | Web Compatibility |
Windows XP | Mihaela | PASS - 2012-09-12 | PASS - 2012-09-13 | PASS - 2012-09-13 |
Windows 7 | Ioana | PASS - 2012-09-12 | PASS - 2012-09-13 | PASS - 2012-09-13 |
Windows 8 | Paul | PASS - 2012-09-12 | PASS - 2012-09-12 | PASS - 2012-09-12 |
Mac OS X | VladG | PASS - 2012-09-12 | PASS - 2012-09-12 | PASS - 2012-09-12 |
Ubuntu 12.04 | Virgil | PASS - 2012-09-12 | PASS - 2012-09-12 | PASS - 2012-09-12 |
Operating System | Tester | Web Mail | Mac 10.8 Signing | Breakpad |
Windows XP | Mihaela | PASS - 2012-09-13 | N/A | PASS - 2012-09-13 |
Windows 7 | Ioana | PASS - 2012-09-13 | N/A | PASS - 2012-09-13 |
Windows 8 | Paul | PASS - 2012-09-13 | N/A | PASS - 2012-09-13 |
Mac OS X | VladG | PASS - 2012-09-12 | PASS - 2012-09-12 | PASS - 2012-09-12 |
Ubuntu 12.04 | Virgil | PASS - 2012-09-12 | N/A | PASS - 2012-09-12 |
Plug-in Compatibility 3
Install the following plug-ins and go to websites which use them, looking for rendering issues, functional issues, performance issues, and crashes.
- Adobe Flash Player
- <a href="http://get.adobe.com/flashplayer/">11.4 Release</a>
- <a href="http://www.microsoft.com/silverlight/">Microsoft Silverlight</a>
- <a href="http://www.real.com/">RealPlayer</a>
- Real Player plugin crashed when opening pages with content managed by the plugin: <a href="http://www.slp3d2.com/launchsiteglobal/test.cfm">click</a>, <a href="http://cf.unc.edu/skillstest/take/realplayer.htm">click</a> - bug 667399
- <a href="http://www.apple.com/quicktime/download/">QuickTime</a>
Add-on Compatibility 3
Test the following websites for rendering issues, functional issues, performance issues, and crashes. Pay particular attention to any problems seen after updating Firefox with add-ons installed.
- <a href="https://addons.mozilla.org/en-US/firefox/compatibility/15.0">Top AMO add-ons</a>
- AVG toolbar (installed via AVG software)
- Norton toolbar (installed via Norton software)
- <a href="http://sg.toolbar.yahoo.com/">Yahoo toolbar</a>
- <a href="http://bar.yandex.ru/en/">Yandex toolbar</a>
- <a href="http://babylon-toolbar.software.informer.com/download/">Babylon toolbar</a>
- <a href="http://lavasoft.com/">Ad-Aware Antivirus</a>
Web Compatibility 3
Test the following websites for rendering issues, functional issues, performance issues, and crashes
- <a href="http://www.alexa.com/topsites">Overall Top 10</a>
- <a href="http://www.alexa.com/topsites/countries">Top by Country</a> (pick a random site from three random countries)
- <a href="http://www.alexa.com/topsites/category">Top by Category</a> (pick a random site from three random categories)
- Webmail clients like gmail, hotmail, and yahoo mail (pay particular attention to any built in web messenger software)
- Try Google's <a href="http://support.google.com/maps/bin/answer.py?hl=en&answer=1630790">MapsGL demo</a>
- Try YouTube's <a href="http://www.youtube.com/html5">HTML5 demo</a>
Web Mail 3
Test the following websites for rendering issues, functional issues, performance issues, and crashes. In particular make sure email, attachments, web messenger, and video chat are working as expected.
- <a href="https://mail.google.com/">Google Mail</a>
- <a href="https://mail.yahoo.com/">Yahoo Mail</a>
- <a href="https://www.hotmail.com">Hotmail</a>
- <a href="https://www.outlook.com/">Outlook.com</a>
Mac 10.8 Signing 3
NOTE: Test needs to be run from a clean Mac OSX 10.8 install. Please keep a vanilla image handy so you can restore once completing this test (<a href="https://people.mozilla.com/~stmichaud/Gatekeeper/MLGatekeeper.html">more info</a>).
- Download the .dmg file and drag the Firefox.app file to a location on your computer
- Open a terminal window and cd to the folder Firefox.app is located
- Run codesign -vv Firefox.app in terminal, output should contain valid on disk and satisfies its Designated Requirement
- Run xattr Firefox.app in terminal, output should contain com.apple.quarantine
- Double click the Firefox.app icon, you should see a warning about the program being provided by an unknown developer or having been downloaded from the Internet
- Click "open" and browse to any website, Firefox should start and you should be able to browse
- Quit Firefox and double click the Firefox.app icon again, you should not be reprompted for the warning in step 5, firefox should start up and browse without any issues
Spot check build after this test
- Try all the menu item functions
- Gestures
- Plug-in compatibility
- Top-site compatibility
- Add-on compatibility
- Software update
Breakpad End-to-End 3
Test the end-to-end functionality in the crash reporting feature, across operating systems.
- Use the <a href="http://ted.mielczarek.org/mozilla/crashme.html">Crash Me add-on</a> to trigger a crash
- Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc
- Repeatedly crash Firefox and submit your reports
- Verify that your session restores properly
- Verify your crash reports are submitted properly by going through about:crashes and clicking the reports
- Try adding comments to your crash report and verify the crash-stats.com report shows the comment
Update Testing 3
- Partial: 16.0b2
- Complete: 15.0b6 and earlier
Platform | Releasetest: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-09-11&to=2012-09-13&target=16.0">report</a> | Beta: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-09-11&to=2012-09-13&target=16.0">report</a> |
Windows XP | PASS - 2012-09-13 | PASS - 2012-09-13 |
Windows Vista | PASS - 2012-09-13 | PASS - 2012-09-13 |
Windows 7 32-bit | PASS - 2012-09-13 | PASS - 2012-09-13 |
Windows 7 64-bit | PASS - 2012-09-13 | PASS - 2012-09-13 |
Windows 8 32-bit | manual-see below - 2012-09-13 | Results - yyyy-mm-dd |
Windows 8 64-bit | manual-see below - 2012-09-13 | Results - yyyy-mm-dd |
Mac OS X 10.6 | PASS - 2012-09-13 | PASS - 2012-09-13 |
Mac OS X 10.7 | PASS - 2012-09-13 | PASS - 2012-09-13 |
Mac OS X 10.8 | PASS - 2012-09-13 | PASS - 2012-09-13 |
Ubuntu 12.04 32-bit | PASS - 2012-09-13 | PASS - 2012-09-13 |
Ubuntu 12.04 64-bit | PASS - 2012-09-13 | PASS - 2012-09-13 |
Windows 8: manual spotcheck until automation in place
- RELEASETEST
- PASS - Firefox 16.0b1 en-US (partial)
- PASS - Firefox 15.0b6 de (complete)
- BETA
- PASS - Firefox 16.0b2 en-US (partial)
- PASS - Firefox 15.0b6 de (complete)
- Issues
Bug Triage 3
- <a href="http://bit.ly/RquQy9">VERIFYME (3)</a> : verify the fix and do some exploratory testing around the issue
- <a href="http://bit.ly/Ppe9E0">FIXED (184)</a> : triage these bugs and test to ensure the fix is effective
- <a href="http://bit.ly/PMaYVx">QAWANTED (8)</a> : triage these bugs and test to fulfill the QA request
- <a href="http://bit.ly/OtEqj1">UNCONFIRMED (10)</a> : triage these bugs for potential new regressions
- Results
- <a href="http://bit.ly/MXYdqS">VERIFIED</a>: 37/249
- <a href="http://bit.ly/POyxuA">WONTVERIFY</a>: 23/249
- <a href="http://bit.ly/O09WSK">HELPWANTED</a>: 5/249
Beta 4
Builds | <a href="ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b4-candidates/build1/">FTP</a>, <a href="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b4-candidates/build1/">HTTP</a> |
ID | 20120919065210 |
Changeset | <a href="http://hg.mozilla.org/releases/mozilla-beta/rev/c3be659f6121">c3be659f6121</a> |
- Checklist
- [DONE] <a href="Releases/Firefox 16/Test Plan#Automated_Testing_4">Automated Testing</a>
- [DONE] <a href="Releases/Firefox 16/Test Plan#Manual_Testing_4">Manual Testing</a>
- [DONE] <a href="Releases/Firefox 16/Test Plan#Update_Testing_4">Update Testing</a>
- [ON TRACK] <a href="Releases/Firefox 16/Test Plan#Bug_Triage_4">Bug Triage</a>
- [DONE] Peer Sign-off
- {{ | }} Final sign-off sent and updates qualified on Beta channel
Automated Testing 4
The following are the detailed results of the automated Mozmill test runs.
Platform | RESULT: <a href="http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-09-18&to=2012-09-19">report</a> |
Windows XP | PASS - 2012-09-19 |
Windows Vista | PASS - 2012-09-19 |
Windows 7 32-bit | PASS - 2012-09-19 |
Windows 7 64-bit | PASS - 2012-09-19 |
Windows 8 32-bit | PASS - 2012-09-19 |
Windows 8 64-bit | PASS - 2012-09-19 |
Mac OS X 10.6 | PASS - 2012-09-19 |
Mac OS X 10.7 | PASS - 2012-09-19 |
Mac OS X 10.8 | PASS - 2012-09-19 |
Ubuntu 12.04 32-bit | PASS - 2012-09-19 |
Ubuntu 12.04 64-bit | PASS - 2012-09-19 |
- Test Failures (<a href="https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA">known bugs</a>)
Manual Testing 4
Operating System | Tester | Plug-in Compatibility | Add-on Compatibility | Web Compatibility |
Windows XP | Vlad, Mihaela | PASS - 2012-09-21 | PASS - 2012-09-21 | PASS - 2012-09-21 |
Windows 7 | Simona | PASS - 2012-09-20 | PASS - 2012-09-20 | PASS - 2012-09-20 |
Windows 8 | Paul | PASS - 2012-09-20 | PASS - 2012-09-20 | PASS - 2012-09-20 |
Mac OS X | VladG | PASS - 2012-09-20 | PASS - 2012-09-20 | PASS - 2012-09-20 |
Ubuntu 12.04 | Mihaela | PASS - 2012-09-20 | PASS - 2012-09-20 | PASS - 2012-09-20 |
Operating System | Tester | Web Mail | Mac 10.8 Signing | Breakpad |
Windows XP | Mihaela | PASS - 2012-09-20 | N/A | PASS - 2012-09-20 |
Windows 7 | Simona | PASS - 2012-09-20 | N/A | PASS - 2012-09-20 |
Windows 8 | Paul | PASS - 2012-09-20 | N/A | PASS - 2012-09-20 |
Mac OS X | VladG | PASS - 2012-09-20 | PASS - 2012-09-20 | PASS - 2012-09-20 |
Ubuntu 12.04 | Mihaela | PASS - 2012-09-20 | N/A | PASS - 2012-09-20 |
Plug-in Compatibility 4
Install the following plug-ins and go to websites which use them, looking for rendering issues, functional issues, performance issues, and crashes.
- Adobe Flash Player
- <a href="http://get.adobe.com/flashplayer/">11.4 Release</a>
- <a href="http://www.microsoft.com/silverlight/">Microsoft Silverlight</a>
- <a href="http://www.real.com/">RealPlayer</a>
- Real Player plugin crashed when opening pages with content managed by the plugin: <a href="http://www.slp3d2.com/launchsiteglobal/test.cfm">click</a>, <a href="http://cf.unc.edu/skillstest/take/realplayer.htm">click</a> - bug 667399
- <a href="http://www.apple.com/quicktime/download/">QuickTime</a>
Add-on Compatibility 4
Test the following websites for rendering issues, functional issues, performance issues, and crashes. Pay particular attention to any problems seen after updating Firefox with add-ons installed.
- <a href="https://addons.mozilla.org/en-US/firefox/compatibility/15.0">Top AMO add-ons</a>
- AVG toolbar (installed via AVG software)
- Norton toolbar (installed via Norton software)
- <a href="http://sg.toolbar.yahoo.com/">Yahoo toolbar</a>
- <a href="http://bar.yandex.ru/en/">Yandex toolbar</a>
- <a href="http://babylon-toolbar.software.informer.com/download/">Babylon toolbar</a>
- Babylon toolbar can't be installed on Firefox 16 - bug 721264
- <a href="http://lavasoft.com/">Ad-Aware Antivirus</a>
Web Compatibility 4
Test the following websites for rendering issues, functional issues, performance issues, and crashes
- <a href="http://www.alexa.com/topsites">Overall Top 10</a>
- <a href="http://www.alexa.com/topsites/countries">Top by Country</a> (pick a random site from three random countries)
- <a href="http://www.alexa.com/topsites/category">Top by Category</a> (pick a random site from three random categories)
- Webmail clients like gmail, hotmail, and yahoo mail (pay particular attention to any built in web messenger software)
- Try Google's <a href="http://support.google.com/maps/bin/answer.py?hl=en&answer=1630790">MapsGL demo</a>
- Try YouTube's <a href="http://www.youtube.com/html5">HTML5 demo</a>
Web Mail 4
Test the following websites for rendering issues, functional issues, performance issues, and crashes. In particular make sure email, attachments, web messenger, and video chat are working as expected.
- <a href="https://mail.google.com/">Google Mail</a>
- <a href="https://mail.yahoo.com/">Yahoo Mail</a>
- <a href="https://www.hotmail.com">Hotmail</a>
- <a href="https://www.outlook.com/">Outlook.com</a>
Mac 10.8 Signing 4
NOTE: Test needs to be run from a clean Mac OSX 10.8 install. Please keep a vanilla image handy so you can restore once completing this test (<a href="https://people.mozilla.com/~stmichaud/Gatekeeper/MLGatekeeper.html">more info</a>).
- Download the .dmg file and drag the Firefox.app file to a location on your computer
- Open a terminal window and cd to the folder Firefox.app is located
- Run codesign -vv Firefox.app in terminal, output should contain valid on disk and satisfies its Designated Requirement
- Run xattr Firefox.app in terminal, output should contain com.apple.quarantine
- Double click the Firefox.app icon, you should see a warning about the program being provided by an unknown developer or having been downloaded from the Internet
- Click "open" and browse to any website, Firefox should start and you should be able to browse
- Quit Firefox and double click the Firefox.app icon again, you should not be reprompted for the warning in step 5, firefox should start up and browse without any issues
Spot check build after this test
- Try all the menu item functions
- Gestures
- Plug-in compatibility
- Top-site compatibility
- Add-on compatibility
- Software update
Breakpad End-to-End 4
Test the end-to-end functionality in the crash reporting feature, across operating systems.
- Use the <a href="http://ted.mielczarek.org/mozilla/crashme.html">Crash Me add-on</a> to trigger a crash
- Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc
- Repeatedly crash Firefox and submit your reports
- Verify that your session restores properly
- Verify your crash reports are submitted properly by going through about:crashes and clicking the reports
- Try adding comments to your crash report and verify the crash-stats.com report shows the comment
Update Testing 4
- Partial: 16.0b3
- Complete: 16.0b2 and earlier
Platform | Releasetest: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-09-18&to=2012-09-19&target=16.0">report</a> | Beta: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-09-18&to=2012-09-19&target=16.0">report</a> |
Windows XP | PASS - 2012-09-20 | RESULT - 2012-09-19 |
Windows Vista | PASS - 2012-09-20 | RESULT - 2012-09-19 |
Windows 7 32-bit | PASS - 2012-09-20 | RESULT - 2012-09-19 |
Windows 7 64-bit | PASS - 2012-09-20 | RESULT - 2012-09-19 |
Windows 8 32-bit | manual-see below - 2012-09-19 | Results - yyyy-mm-dd |
Windows 8 64-bit | manual-see below - 2012-09-19 | Results - yyyy-mm-dd |
Mac OS X 10.6 | PASS - 2012-09-20 | RESULT - 2012-09-19 |
Mac OS X 10.7 | PASS - 2012-09-20 | RESULT - 2012-09-19 |
Mac OS X 10.8 | PASS - 2012-09-20 | RESULT - 2012-09-19 |
Ubuntu 12.04 32-bit | PASS - 2012-09-20 | RESULT - 2012-09-19 |
Ubuntu 12.04 64-bit | PASS - 2012-09-20 | RESULT - 2012-09-19 |
Windows 8: manual spotcheck until automation in place
- RELEASETEST
- PASS - Firefox 16.0b3 en-US (partial)
- PASS - Firefox 15.0b6 de (complete)
- BETA
- RESULT - Firefox 16.0b3 en-US (partial)
- RESULT - Firefox 15.0b6 de (complete)
- Issues
Bug Triage 4
- <a href="http://bit.ly/RquQy9">VERIFYME (?)</a> : verify the fix and do some exploratory testing around the issue
- <a href="http://bit.ly/Ppe9E0">FIXED (?)</a> : triage these bugs and test to ensure the fix is effective
- <a href="http://bit.ly/PMaYVx">QAWANTED (?)</a> : triage these bugs and test to fulfill the QA request
- <a href="http://bit.ly/OtEqj1">UNCONFIRMED (?)</a> : triage these bugs for potential new regressions
- Results
- <a href="http://bit.ly/MXYdqS">VERIFIED</a>: ratios
- <a href="http://bit.ly/POyxuA">WONTVERIFY</a>: ratios
- <a href="http://bit.ly/O09WSK">HELPWANTED</a>: ratios
Verification <a href="https://docs.google.com/spreadsheet/ccc?key=0AoLS2jzcUzsHdEFLcDlFa1ppTWZXRDBvZ3Btamk2MVE#gid=0">Spreadsheet</a>
Beta 5
Builds | <a href="ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b5-candidates/build1/">FTP</a>, <a href="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b5-candidates/build1/">HTTP</a> |
ID | 20120925201946 |
Changeset | <a href="https://hg.mozilla.org/releases/mozilla-beta/rev/7c0af9b7ed61">7c0af9b7ed61</a> (<a href="http://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=FIREFOX_16_0b4_BUILD1&tochange=FIREFOX_16_0b5_BUILD1">pushlog</a>) |
- Checklist
- [DONE] <a href="Releases/Firefox 16/Test Plan#Automated_Testing_5">Automated Testing</a>
- [DONE] <a href="Releases/Firefox 16/Test Plan#Manual_Testing_5">Manual Testing</a>
- [DONE] <a href="Releases/Firefox 16/Test Plan#Update_Testing_5">Update Testing</a>
- [ON TRACK] <a href="Releases/Firefox 16/Test Plan#Bug_Triage_5">Bug Triage</a>
- [DONE] Peer Sign-off
- {{ | }} Final sign-off sent and updates qualified on Beta channel
Automated Testing 5
The following are the detailed results of the automated Mozmill test runs.
Platform | RESULT: <a href="http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-09-25&to=2012-09-26">report</a> |
Windows XP | PASS - 2012-09-26 |
Windows Vista | PASS - 2012-09-26 |
Windows 7 32-bit | PASS - 2012-09-26 |
Windows 7 64-bit | PASS - 2012-09-26 |
Windows 8 32-bit | PASS - 2012-09-26 |
Windows 8 64-bit | PASS - 2012-09-26 |
Mac OS X 10.6 | PASS - 2012-09-26 |
Mac OS X 10.7 | PASS - 2012-09-26 |
Mac OS X 10.8 | PASS - 2012-09-26 |
Ubuntu 12.04 32-bit | PASS - 2012-09-26 |
Ubuntu 12.04 64-bit | PASS - 2012-09-26 |
- Test Failures (<a href="https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA">known bugs</a>)
Manual Testing 5
Operating System | Tester | Plug-in Compatibility | Add-on Compatibility | Web Compatibility |
Windows XP | Simona | PASS - 2012-09-26 | PASS - 2012-09-26 | PASS - 2012-09-26 |
Windows 7 | Paul | PASS - 2012-09-26 | PASS - 2012-09-26 | PASS - 2012-09-26 |
Windows 8 | Mihaela | PASS - 2012-09-26 | PASS - 2012-09-26 | PASS - 2012-09-26 |
Mac OS X | VladG | PASS - 2012-09-26 | PASS - 2012-09-26 | PASS - 2012-09-26 |
Ubuntu 12.04 | Ioana | PASS - 2012-09-26 | PASS - 2012-09-26 | PASS - 2012-09-27 |
Operating System | Tester | Web Mail | Mac 10.8 Signing | Breakpad |
Windows XP | Simona | PASS - 2012-09-26 | N/A | PASS - 2012-09-26 |
Windows 7 | Paul | PASS - 2012-09-26 | N/A | PASS - 2012-09-26 |
Windows 8 | Mihaela | PASS - 2012-09-26 | N/A | PASS - 2012-09-27 |
Mac OS X | VladG | PASS - 2012-09-27 | PASS - 2012-09-26 | PASS - 2012-09-27 |
Ubuntu 12.04 | Ioana | PASS - 2012-09-27 | N/A | PASS - 2012-09-27 |
Plug-in Compatibility 5
Install the following plug-ins and go to websites which use them, looking for rendering issues, functional issues, performance issues, and crashes.
- Adobe Flash Player
- <a href="http://get.adobe.com/flashplayer/">11.4 Release</a>
- <a href="http://www.microsoft.com/silverlight/">Microsoft Silverlight</a>
- <a href="http://www.real.com/">RealPlayer</a>
- Real Player plugin crashed when opening pages with content managed by the plugin: <a href="http://www.slp3d2.com/launchsiteglobal/test.cfm">click</a>, <a href="http://cf.unc.edu/skillstest/take/realplayer.htm">click</a> - bug 667399
- <a href="http://www.apple.com/quicktime/download/">QuickTime</a>
Add-on Compatibility 5
Test the following websites for rendering issues, functional issues, performance issues, and crashes. Pay particular attention to any problems seen after updating Firefox with add-ons installed.
- <a href="https://addons.mozilla.org/en-US/firefox/compatibility/16.0">Top AMO add-ons</a>
- AVG toolbar (installed via AVG software)
- Norton toolbar (installed via Norton software)
- <a href="http://sg.toolbar.yahoo.com/">Yahoo toolbar</a>
- <a href="http://bar.yandex.ru/en/">Yandex toolbar</a>
- <a href="http://babylon-toolbar.software.informer.com/download/">Babylon toolbar</a>
- Babylon toolbar can't be installed on Firefox 16 - bug 721264
- <a href="http://lavasoft.com/">Ad-Aware Antivirus</a>
Web Compatibility 5
Test the following websites for rendering issues, functional issues, performance issues, and crashes
- <a href="http://www.alexa.com/topsites">Overall Top 10</a>
- <a href="http://www.alexa.com/topsites/countries">Top by Country</a> (pick a random site from three random countries)
- <a href="http://www.alexa.com/topsites/category">Top by Category</a> (pick a random site from three random categories)
- Webmail clients like gmail, hotmail, and yahoo mail (pay particular attention to any built in web messenger software)
- Try Google's <a href="http://support.google.com/maps/bin/answer.py?hl=en&answer=1630790">MapsGL demo</a>
- Try YouTube's <a href="http://www.youtube.com/html5">HTML5 demo</a>
Web Mail 5
Test the following websites for rendering issues, functional issues, performance issues, and crashes. In particular make sure email, attachments, web messenger, and video chat are working as expected.
- <a href="https://mail.google.com/">Google Mail</a>
- <a href="https://mail.yahoo.com/">Yahoo Mail</a>
- <a href="https://www.hotmail.com">Hotmail</a>
- <a href="https://www.outlook.com/">Outlook.com</a>
Mac 10.8 Signing 5
NOTE: Test needs to be run from a clean Mac OSX 10.8 install. Please keep a vanilla image handy so you can restore once completing this test (<a href="https://people.mozilla.com/~stmichaud/Gatekeeper/MLGatekeeper.html">more info</a>).
- Download the .dmg file and drag the Firefox.app file to a location on your computer
- Open a terminal window and cd to the folder Firefox.app is located
- Run codesign -vv Firefox.app in terminal, output should contain valid on disk and satisfies its Designated Requirement
- Run xattr Firefox.app in terminal, output should contain com.apple.quarantine
- Double click the Firefox.app icon, you should see a warning about the program being provided by an unknown developer or having been downloaded from the Internet
- Click "open" and browse to any website, Firefox should start and you should be able to browse
- Quit Firefox and double click the Firefox.app icon again, you should not be reprompted for the warning in step 5, firefox should start up and browse without any issues
Spot check build after this test
- Try all the menu item functions
- Gestures
- Plug-in compatibility
- Top-site compatibility
- Add-on compatibility
- Software update
Breakpad End-to-End 5
Test the end-to-end functionality in the crash reporting feature, across operating systems.
- Use the <a href="http://ted.mielczarek.org/mozilla/crashme.html">Crash Me add-on</a> to trigger a crash
- Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc
- Repeatedly crash Firefox and submit your reports
- Verify that your session restores properly
- Verify your crash reports are submitted properly by going through about:crashes and clicking the reports
- Try adding comments to your crash report and verify the crash-stats.com report shows the comment
Windows 8
Test Firefox integration with Windows 8 - <a href="https://etherpad.mozilla.org/Win-8-Classic-Test-Plan">etherpad</a>.
Update Testing 5
- Partial: 16.0b4
- Complete: 16.0b3 and earlier
Platform | Releasetest: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-09-25&to=2012-09-26&target=16.0">report</a> | Beta: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-09-25&to=2012-09-26&target=16.0">report</a> |
Windows XP | PASS - 2012-09-26 | RESULT - 2012-09-26 |
Windows Vista | PASS - 2012-09-26 | RESULT - 2012-09-26 |
Windows 7 32-bit | PASS - 2012-09-26 | RESULT - 2012-09-26 |
Windows 7 64-bit | PASS - 2012-09-26 | RESULT - 2012-09-26 |
Windows 8 32-bit | manual-see below - 2012-09-19 | Results - yyyy-mm-dd |
Windows 8 64-bit | manual-see below - 2012-09-19 | Results - yyyy-mm-dd |
Mac OS X 10.6 | PASS - 2012-09-26 | RESULT - 2012-09-26 |
Mac OS X 10.7 | PASS - 2012-09-26 | RESULT - 2012-09-26 |
Mac OS X 10.8 | PASS - 2012-09-26 | RESULT - 2012-09-26 |
Ubuntu 12.04 32-bit | PASS - 2012-09-26 | RESULT - 2012-09-26 |
Ubuntu 12.04 64-bit | PASS - 2012-09-26 | RESULT - 2012-09-26 |
Windows 8: manual spotcheck until automation in place
- RELEASETEST
- RESULT - Firefox 16.0b4 en-US (partial)
- RESULT - Firefox 15.0b6 de (complete)
- BETA
- RESULT - Firefox 16.0b4 en-US (partial)
- RESULT - Firefox 15.0b6 de (complete)
- Issues
Bug Triage 5
- <a href="http://bit.ly/RquQy9">VERIFYME (19)</a> : verify the fix and do some exploratory testing around the issue
- <a href="http://bit.ly/Ppe9E0">FIXED (184)</a> : triage these bugs and test to ensure the fix is effective
- <a href="http://bit.ly/PMaYVx">QAWANTED (6)</a> : triage these bugs and test to fulfill the QA request
- <a href="http://bit.ly/OtEqj1">UNCONFIRMED (14)</a> : triage these bugs for potential new regressions
- Results
- <a href="http://bit.ly/MXYdqS">VERIFIED</a>: 81/296
- <a href="http://bit.ly/POyxuA">WONTVERIFY</a>: 30/296
- <a href="http://bit.ly/O09WSK">HELPWANTED</a>: 5/296
Beta 6
Builds | <a href="ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b6-candidates/build1/">FTP</a>, <a href="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0b6-candidates/build1/">HTTP</a> |
ID | 20121002073616 |
Changeset | <a href="https://hg.mozilla.org/releases/mozilla-beta/rev/8a8a932fbef4">8a8a932fbef4</a> (<a href="http://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=FIREFOX_16_0b5_BUILD1&tochange=FIREFOX_16_0b6_BUILD1">pushlog</a>) |
- Checklist
- [DONE] <a href="Releases/Firefox 16/Test Plan#Automated_Testing_6">Automated Testing</a>
- [DONE] <a href="Releases/Firefox 16/Test Plan#Manual_Testing_6">Manual Testing</a>
- [DONE] <a href="Releases/Firefox 16/Test Plan#Update_Testing_6">Update Testing</a>
- [ON TRACK] <a href="Releases/Firefox 16/Test Plan#Bug_Triage_6">Bug Triage</a>
- [ON TRACK] Peer Sign-off
- {{ | }} Final sign-off sent and updates qualified on Beta channel
Automated Testing 6
The following are the detailed results of the automated Mozmill test runs.
Platform | RESULT: <a href="http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-10-01&to=2012-10-02">report</a> |
Windows XP | PASS - 2012-10-02 |
Windows Vista | PASS - 2012-10-02 |
Windows 7 32-bit | PASS - 2012-10-02 |
Windows 7 64-bit | PASS - 2012-10-02 |
Windows 8 32-bit | PASS - 2012-10-02 |
Windows 8 64-bit | PASS - 2012-10-02 |
Mac OS X 10.6 | PASS - 2012-10-02 |
Mac OS X 10.7 | PASS - 2012-10-02 |
Mac OS X 10.8 | PASS - 2012-10-02 |
Ubuntu 12.04 32-bit | PASS - 2012-10-02 |
Ubuntu 12.04 64-bit | PASS - 2012-10-02 |
- Test Failures (<a href="https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA">known bugs</a>)
Manual Testing 6
Operating System | Tester | Plug-in Compatibility | Add-on Compatibility | Web Compatibility |
Windows XP | Paul/Virgil | PASS - 2012-10-02 | PASS - 2012-10-02 | PASS - 2012-10-02 |
Windows 7 | Ioana | PASS - 2012-10-02 | PASS - 2012-10-02 | PASS - 2012-10-02 |
Windows 8 | Simona | PASS - 2012-10-02 | PASS - 2012-10-02 | PASS - 2012-10-02 |
Mac OS X | VladG | PASS - 2012-10-03 | PASS - 2012-10-03 | PASS - 2012-10-03 |
Ubuntu 12.04 | Mihaela | PASS - 2012-10-02 | PASS - 2012-10-02 | PASS - 2012-10-02 |
Operating System | Tester | Web Mail | Mac 10.8 Signing | Breakpad |
Windows XP | Paul/Virgil | PASS - 2012-10-02 | N/A | PASS - 2012-10-02 |
Windows 7 | Ioana/Paul | PASS - 2012-10-02 | N/A | PASS - 2012-10-02 |
Windows 8 | Simona | PASS - 2012-10-02 | N/A | PASS - 2012-10-02 |
Mac OS X | VladG | PASS - 2012-10-03 | PASS - 2012-10-03 | PASS - 2012-10-03 |
Ubuntu 12.04 | Mihaela | PASS - 2012-10-02 | N/A | PASS - 2012-10-02 |
Plug-in Compatibility 6
Install the following plug-ins and go to websites which use them, looking for rendering issues, functional issues, performance issues, and crashes.
- Adobe Flash Player
- <a href="http://get.adobe.com/flashplayer/">11.4 Release</a>
- <a href="http://www.microsoft.com/silverlight/">Microsoft Silverlight</a>
- <a href="http://www.real.com/">RealPlayer</a>
- Real Player plugin crashed when opening pages with content managed by the plugin: <a href="http://www.slp3d2.com/launchsiteglobal/test.cfm">click</a>, <a href="http://cf.unc.edu/skillstest/take/realplayer.htm">click</a> - bug 667399
- <a href="http://www.apple.com/quicktime/download/">QuickTime</a>
Add-on Compatibility 6
Test the following websites for rendering issues, functional issues, performance issues, and crashes. Pay particular attention to any problems seen after updating Firefox with add-ons installed.
- <a href="https://addons.mozilla.org/en-US/firefox/compatibility/16.0">Top AMO add-ons</a>
- AVG toolbar (installed via AVG software)
- Norton toolbar (installed via Norton software)
- <a href="http://sg.toolbar.yahoo.com/">Yahoo toolbar</a>
- <a href="http://bar.yandex.ru/en/">Yandex toolbar</a>
- <a href="http://babylon-toolbar.software.informer.com/download/">Babylon toolbar</a>
- <a href="http://lavasoft.com/">Ad-Aware Antivirus</a>
Web Compatibility 6
Test the following websites for rendering issues, functional issues, performance issues, and crashes
- <a href="http://www.alexa.com/topsites">Overall Top 10</a>
- <a href="http://www.alexa.com/topsites/countries">Top by Country</a> (pick a random site from three random countries)
- <a href="http://www.alexa.com/topsites/category">Top by Category</a> (pick a random site from three random categories)
- Webmail clients like gmail, hotmail, and yahoo mail (pay particular attention to any built in web messenger software)
- Try Google's <a href="http://support.google.com/maps/bin/answer.py?hl=en&answer=1630790">MapsGL demo</a>
- Try YouTube's <a href="http://www.youtube.com/html5">HTML5 demo</a>
Web Mail 6
Test the following websites for rendering issues, functional issues, performance issues, and crashes. In particular make sure email, attachments, web messenger, and video chat are working as expected.
- <a href="https://mail.google.com/">Google Mail</a>
- <a href="https://mail.yahoo.com/">Yahoo Mail</a>
- <a href="https://www.hotmail.com">Hotmail</a>
- <a href="https://www.outlook.com/">Outlook.com</a>
Mac 10.8 Signing 6
NOTE: Test needs to be run from a clean Mac OSX 10.8 install. Please keep a vanilla image handy so you can restore once completing this test (<a href="https://people.mozilla.com/~stmichaud/Gatekeeper/MLGatekeeper.html">more info</a>).
- Download the .dmg file and drag the Firefox.app file to a location on your computer
- Open a terminal window and cd to the folder Firefox.app is located
- Run codesign -vv Firefox.app in terminal, output should contain valid on disk and satisfies its Designated Requirement
- Run xattr Firefox.app in terminal, output should contain com.apple.quarantine
- Double click the Firefox.app icon, you should see a warning about the program being provided by an unknown developer or having been downloaded from the Internet
- Click "open" and browse to any website, Firefox should start and you should be able to browse
- Quit Firefox and double click the Firefox.app icon again, you should not be reprompted for the warning in step 5, firefox should start up and browse without any issues
Spot check build after this test
- Try all the menu item functions
- Gestures
- Plug-in compatibility
- Top-site compatibility
- Add-on compatibility
- Software update
Breakpad End-to-End 6
Test the end-to-end functionality in the crash reporting feature, across operating systems.
- Use the <a href="http://ted.mielczarek.org/mozilla/crashme.html">Crash Me add-on</a> to trigger a crash
- Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc
- Repeatedly crash Firefox and submit your reports
- Verify that your session restores properly
- Verify your crash reports are submitted properly by going through about:crashes and clicking the reports
- Try adding comments to your crash report and verify the crash-stats.com report shows the comment
Update Testing 6
- Partial: 16.0b5
- Complete: 16.0b4 and earlier
Platform | Releasetest: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-10-02&to=2012-10-03&target=16.0">report</a> | Beta: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-10-02&to=2012-10-03&target=16.0">report</a> |
Windows XP | PASS - 2012-10-02 | RESULT - 2012-10-02 |
Windows Vista | PASS - 2012-10-02 | RESULT - 2012-10-02 |
Windows 7 32-bit | PASS - 2012-10-02 | RESULT - 2012-10-02 |
Windows 7 64-bit | PASS - 2012-10-02 | RESULT - 2012-10-02 |
Windows 8 32-bit | manual-see below - 2012-10-02 | Results - yyyy-mm-dd |
Windows 8 64-bit | manual-see below - 2012-10-02 | Results - yyyy-mm-dd |
Mac OS X 10.6 | PASS - 2012-10-02 | RESULT - 2012-10-02 |
Mac OS X 10.7 | PASS - 2012-10-02 | RESULT - 2012-10-02 |
Mac OS X 10.8 | PASS - 2012-10-02 | RESULT - 2012-10-02 |
Ubuntu 12.04 32-bit | PASS - 2012-10-02 | RESULT - 2012-10-02 |
Ubuntu 12.04 64-bit | PASS - 2012-10-02 | RESULT - 2012-10-02 |
Windows 8: manual spotcheck until automation in place
- RELEASETEST
- RESULT - Firefox 16.0b5 en-US (partial)
- RESULT - Firefox 15.0b6 de (complete)
- BETA
- RESULT - Firefox 16.0b5 en-US (partial)
- RESULT - Firefox 15.0b6 de (complete)
- Issues
Bug Triage 6
- <a href="http://bit.ly/RquQy9">VERIFYME (10)</a> : verify the fix and do some exploratory testing around the issue
- <a href="http://bit.ly/Ppe9E0">FIXED (181)</a> : triage these bugs and test to ensure the fix is effective
- <a href="http://bit.ly/PMaYVx">QAWANTED (4)</a> : triage these bugs and test to fulfill the QA request
- <a href="http://bit.ly/OtEqj1">UNCONFIRMED (18)</a> : triage these bugs for potential new regressions
- Results
- <a href="http://bit.ly/MXYdqS">VERIFIED</a>: 87/303
- <a href="http://bit.ly/POyxuA">WONTVERIFY</a>: 31/303
- <a href="http://bit.ly/O09WSK">HELPWANTED</a>: 4/303
Release Candidate
Builds | <a href="ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0-candidates/build1/">FTP</a>, <a href="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0-candidates/build1/">HTTP</a> |
ID | 20121005155445 |
Changeset | <a href="http://hg.mozilla.org/releases/mozilla-release/rev/10fe550fadc6">10fe550fadc6</a> |
- Checklist
- [DONE] <a href="Releases/Firefox 16/Test Plan#Automated_Testing_RC">Automated Testing</a>
- [DONE] <a href="Releases/Firefox 16/Test Plan#Manual_Testing_RC">Manual Testing</a>
- [DONE] <a href="Releases/Firefox 16/Test Plan#Update_Testing_RC">Update Testing</a>
- [ON TRACK] <a href="Releases/Firefox 16/Test Plan#Bug_Triage_RC">Bug Triage</a>
- {{|}} Peer Sign-off
- {{ | }} Final sign-off sent and updates qualified on Release channel
Automated Testing RC
The following are the detailed results of the automated Mozmill test runs.
Platform | RESULT: <a href="http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-10-05&to=2012-10-09">report</a> |
Windows XP | PASS - 2012-10-08 |
Windows Vista | PASS - 2012-10-08 |
Windows 7 32-bit | PASS - 2012-10-08 |
Windows 7 64-bit | PASS - 2012-10-08 |
Windows 8 32-bit | PASS - 2012-10-08 |
Windows 8 64-bit | PASS - 2012-10-08 |
Mac OS X 10.6 | PASS - 2012-10-08 |
Mac OS X 10.7 | PASS - 2012-10-08 |
Mac OS X 10.8 | PASS - 2012-10-08 |
Ubuntu 12.04 32-bit | PASS - 2012-10-08 |
Ubuntu 12.04 64-bit | PASS - 2012-10-08 |
- Test Failures (<a href="https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA">known bugs</a>)
Manual Testing RC
Operating System | Tester | Plug-in Compatibility | Add-on Compatibility | Web Compatibility |
Windows XP | Ioana | PASS - 2012-10-09 | PASS - 2012-10-09 | PASS - 2012-10-09 |
Windows 7 | Simona | PASS - 2012-10-08 | PASS - 2012-10-09 | PASS - 2012-10-09 |
Windows 8 | Mihaela | PASS - 2012-10-09 | PASS - 2012-10-09 | PASS - 2012-10-09 |
Mac OS X | VladG | PASS - 2012-10-09 | PASS - 2012-10-09 | PASS - 2012-10-09 |
Ubuntu 12.04 | Virgil | PASS - 2012-10-08 | PASS - 2012-10-09 | PASS - 2012-10-08 |
Operating System | Tester | Web Mail | Mac 10.8 Signing | Breakpad |
Windows XP | Paul | PASS - 2012-10-09 | N/A | PASS - 2012-10-09 |
Windows 7 | Simona | PASS - 2012-10-09 | N/A | PASS - 2012-10-09 |
Windows 8 | Mihaela | PASS - 2012-10-09 | N/A | PASS - 2012-10-09 |
Mac OS X | VladG | PASS - 2012-10-09 | PASS - 2012-10-08 | PASS - 2012-10-09 |
Ubuntu 12.04 | Virgil | PASS - 2012-10-08 | N/A | PASS - 2012-10-09 |
Plug-in Compatibility
Install the following plug-ins and go to websites which use them, looking for rendering issues, functional issues, performance issues, and crashes.
- Adobe Flash Player
- <a href="http://get.adobe.com/flashplayer/">11.4 Release</a>
- <a href="http://www.microsoft.com/silverlight/">Microsoft Silverlight</a>
- <a href="http://www.real.com/">RealPlayer</a>
- Real Player plugin crashed when opening pages with content managed by the plugin: <a href="http://www.slp3d2.com/launchsiteglobal/test.cfm">click</a>, <a href="http://cf.unc.edu/skillstest/take/realplayer.htm">click</a> - bug 667399
- <a href="http://www.apple.com/quicktime/download/">QuickTime</a>
Add-on Compatibility
Test the following websites for rendering issues, functional issues, performance issues, and crashes. Pay particular attention to any problems seen after updating Firefox with add-ons installed.
- <a href="https://addons.mozilla.org/en-US/firefox/compatibility/16.0">Top AMO add-ons</a>
- AVG toolbar (installed via AVG software)
- Norton toolbar (installed via Norton software)
- <a href="http://sg.toolbar.yahoo.com/">Yahoo toolbar</a>
- <a href="http://bar.yandex.ru/en/">Yandex toolbar</a>
- <a href="http://babylon-toolbar.software.informer.com/download/">Babylon toolbar</a>
- <a href="http://lavasoft.com/">Ad-Aware Antivirus</a>
Web Compatibility
Test the following websites for rendering issues, functional issues, performance issues, and crashes
- <a href="http://www.alexa.com/topsites">Overall Top 10</a>
- <a href="http://www.alexa.com/topsites/countries">Top by Country</a> (pick a random site from three random countries)
- <a href="http://www.alexa.com/topsites/category">Top by Category</a> (pick a random site from three random categories)
- Webmail clients like gmail, hotmail, and yahoo mail (pay particular attention to any built in web messenger software)
- Try Google's <a href="http://support.google.com/maps/bin/answer.py?hl=en&answer=1630790">MapsGL demo</a>
- Try YouTube's <a href="http://www.youtube.com/html5">HTML5 demo</a>
Web Mail
Test the following websites for rendering issues, functional issues, performance issues, and crashes. In particular make sure email, attachments, web messenger, and video chat are working as expected.
- <a href="https://mail.google.com/">Google Mail</a>
- <a href="https://mail.yahoo.com/">Yahoo Mail</a>
- <a href="https://www.hotmail.com">Hotmail</a>
- <a href="https://www.outlook.com/">Outlook.com</a>
Mac 10.8 Signing
NOTE: Test needs to be run from a clean Mac OSX 10.8 install. Please keep a vanilla image handy so you can restore once completing this test (<a href="https://people.mozilla.com/~stmichaud/Gatekeeper/MLGatekeeper.html">more info</a>).
- Download the .dmg file and drag the Firefox.app file to a location on your computer
- Open a terminal window and cd to the folder Firefox.app is located
- Run codesign -vv Firefox.app in terminal, output should contain valid on disk and satisfies its Designated Requirement
- Run xattr Firefox.app in terminal, output should contain com.apple.quarantine
- Double click the Firefox.app icon, you should see a warning about the program being provided by an unknown developer or having been downloaded from the Internet
- Click "open" and browse to any website, Firefox should start and you should be able to browse
- Quit Firefox and double click the Firefox.app icon again, you should not be reprompted for the warning in step 5, firefox should start up and browse without any issues
Spot check build after this test
- Try all the menu item functions
- Gestures
- Plug-in compatibility
- Top-site compatibility
- Add-on compatibility
- Software update
Breakpad End-to-End
Test the end-to-end functionality in the crash reporting feature, across operating systems.
- Use the <a href="http://ted.mielczarek.org/mozilla/crashme.html">Crash Me add-on</a> to trigger a crash
- Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc
- Repeatedly crash Firefox and submit your reports
- Verify that your session restores properly
- Verify your crash reports are submitted properly by going through about:crashes and clicking the reports
- Try adding comments to your crash report and verify the crash-stats.com report shows the comment
Update Testing RC
- Partial: 15.0.1
- Complete: 12.0 and earlier
Platform | Releasetest: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-10-08&to=2012-10-10&target=16.0">report</a> | Beta: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-10-08&to=2012-10-10&target=16.0">report</a> |
Windows XP | PASS - 2010-10-08 | RESULTS - yyyy-mm-dd |
Windows Vista | PASS - 2010-10-08 | RESULTS - yyyy-mm-dd |
Windows 7 32-bit | PASS - 2010-10-08 | RESULTS - yyyy-mm-dd |
Windows 7 64-bit | PASS - 2010-10-08 | RESULTS - yyyy-mm-dd |
Windows 8 32-bit | PASS- see below - 2012-10-09 | RESULTS - yyyy-mm-dd |
Windows 8 64-bit | PASS- see below - 2012-10-08 | RESULTS - yyyy-mm-dd |
Mac OS X 10.6 | PASS - 2010-10-08 | RESULTS - yyyy-mm-dd |
Mac OS X 10.7 | PASS - 2010-10-08 | RESULTS - yyyy-mm-dd |
Mac OS X 10.8 | PASS - 2010-10-08 | RESULTS - yyyy-mm-dd |
Ubuntu 12.04 32-bit | PASS - 2010-10-08 | RESULTS - yyyy-mm-dd |
Ubuntu 12.04 64-bit | PASS - 2010-10-08 | RESULTS - yyyy-mm-dd |
Windows 8: manual spotcheck until automation in place
- BETATEST
- PASS - Firefox 15.0.1 en-US de (partial)
- PASS - Firefox 12.0 en-US fr (complete)
- RELEASE
- RESULT - Firefox 15.0.1 en-US de (partial)
- RESULT - Firefox 12.0 en-US de (complete)
- Issues
Bug Triage RC
- <a href="http://bit.ly/RquQy9">VERIFYME (9)</a> : verify the fix and do some exploratory testing around the issue
- <a href="http://bit.ly/Ppe9E0">FIXED (178)</a> : triage these bugs and test to ensure the fix is effective
- <a href="http://bit.ly/PMaYVx">QAWANTED (8)</a> : triage these bugs and test to fulfill the QA request
- <a href="http://bit.ly/OtEqj1">UNCONFIRMED (21)</a> : triage these bugs for potential new regressions
- Results
- <a href="http://bit.ly/MXYdqS">VERIFIED</a>: 89/304
- <a href="http://bit.ly/POyxuA">WONTVERIFY</a>: 32/304
- <a href="http://bit.ly/O09WSK">HELPWANTED</a>: 5/304
16.0.1
Builds | <a href="ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0.1-candidates/build1/">FTP</a>, <a href="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0.1-candidates/build1/">HTTP</a> |
ID | 20121010144125 |
Changeset | <a href="https://hg.mozilla.org/releases/mozilla-release/rev/7789ab68a99d">7789ab68a99d</a> (<a href="http://hg.mozilla.org/releases/mozilla-release/pushloghtml?fromchange=10fe550fadc6&tochange=7789ab68a99d">pushlog</a>) |
- Checklist
- [DONE] <a href="Releases/Firefox 16/Test Plan#Automated_Testing_7">Automated Testing</a>
- [DONE] <a href="Releases/Firefox 16/Test Plan#Manual_Testing_7">Manual Testing</a>
- [DONE] <a href="Releases/Firefox 16/Test Plan#Update_Testing_7">Update Testing</a>
- {{ | }} Final sign-off sent and updates qualified on Release channel
Automated Testing 7
The following are the detailed results of the automated Mozmill test runs.
Platform | RESULT: <a href="http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-10-10&to=2012-10-11">report</a> |
Windows XP | PASS - 2012-10-10 |
Windows Vista | PASS - 2012-10-10 |
Windows 7 32-bit | PASS - 2012-10-10 |
Windows 7 64-bit | PASS - 2012-10-10 |
Windows 8 32-bit | PASS - 2012-10-10 |
Windows 8 64-bit | PASS - 2012-10-10 |
Mac OS X 10.6 | PASS - 2012-10-10 |
Mac OS X 10.7 | PASS - 2012-10-10 |
Mac OS X 10.8 | PASS - 2012-10-10 |
Ubuntu 12.04 32-bit | PASS - 2012-10-10 |
Ubuntu 12.04 64-bit | PASS - 2012-10-10 |
- Test Failures (<a href="https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA">known bugs</a>)
Manual Testing 7
There will be four bug verifications and testing around these bugs
- Results
Platform | Tester | 16.0.1 Candidates |
Windows XP | juanb, ashughes, mwobensmith | PASS - 2012-10-10 |
Windows Vista | tester | RESULT - 2012-10-10 |
Windows 7 32-bit | tester | RESULT - 2012-10-10 |
Windows 7 64-bit | juanb, ashughes, mwobensmith | PASS - 2012-10-10 |
Mac OS X 10.6 | tester | RESULT - 2012-10-10 |
Mac OS X 10.7 | juanb, ashughes, mwobensmith | PASS - 2012-10-10 |
Mac OS X 10.8 | tester | RESULT - 2012-10-10 |
Ubuntu 12.04 32-bit | tester | RESULT - 2012-10-10 |
Ubuntu 12.04 64-bit | juanb, ashughes, mwobensmith | PASS - 2012-10-10 |
- Issues Found
Breakpad End-to-End 7
Test the end-to-end functionality in the crash reporting feature, across operating systems.
- Use the <a href="http://ted.mielczarek.org/mozilla/crashme.html">Crash Me add-on</a> to trigger a crash
- Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc
- Repeatedly crash Firefox and submit your reports
- Verify that your session restores properly
- Verify your crash reports are submitted properly by going through about:crashes and clicking the reports
- Try adding comments to your crash report and verify the crash-stats.com report shows the comment
Update Testing 7
- Partial: 13.0.1, 14.0.1, 15.0.1
- Complete: 12.0, 13.0
Platform | Releasetest: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-10-10&to=2012-10-10&target=16.0">report</a> | Release: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-10-11&to=2012-10-11&target=16.0">report</a> |
Windows XP | PASS - 2012-10-11 | PASS - 2012-10-11 |
Windows Vista | PASS - 2012-10-11 | PASS - 2012-10-11 |
Windows 7 32-bit | PASS - 2012-10-11 | PASS - 2012-10-11 |
Windows 7 64-bit | PASS - 2012-10-11 | PASS - 2012-10-11 |
Windows 8 32-bit | Pass- juanb - 2012-10-11 | Pass - juanb - 2012-10-11 |
Windows 8 64-bit | Pass- juanb - 2012-10-11 | Pass - 2012-10-11 |
Mac OS X 10.6 | PASS - 2012-10-11 | PASS - 2012-10-11 |
Mac OS X 10.7 | PASS - 2012-10-11 | PASS - 2012-10-11 |
Mac OS X 10.8 | PASS - 2012-10-11 | PASS - 2012-10-11 |
Ubuntu 12.04 32-bit | PASS - 2012-10-11 | PASS - 2012-10-11 |
Ubuntu 12.04 64-bit | PASS - 2012-10-11 | PASS - 2012-10-11 |
Windows 8: manual spotcheck until automation in place
- RELEASETEST
- PASS - Firefox 16.0 en-US (partial)
- PASS - Firefox 15.0.1 de (partial)
- RELEASE
- PASS - Firefox 16.0 en-US (partial)
- PASS - Firefox 12.0 fr (complete)
- Issues
16.0.2
Builds | <a href="ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0.2-candidates/build1/">FTP</a>, <a href="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0.2-candidates/build1/">HTTP</a> |
ID | 20121024073032 |
Changeset | <a href="https://hg.mozilla.org/releases/mozilla-release/rev/3995ff235342">3995ff235342</a> (<a href="http://hg.mozilla.org/releases/mozilla-release/pushloghtml?fromchange=7789ab68a99d&tochange=3995ff235342">pushlog</a>) |
- Checklist
- [DONE] <a href="Releases/Firefox 16/Test Plan#Automated_Testing_8">Automated Testing</a>
- [ON TRACK] <a href="Releases/Firefox 16/Test Plan#Manual_Testing_8">Manual Testing</a>
- [ON TRACK] <a href="Releases/Firefox 16/Test Plan#Update_Testing_8">Update Testing</a>
- {{ | }} Final sign-off sent and updates qualified on Beta channel
Automated Testing 8
The following are the detailed results of the automated Mozmill test runs.
Platform | RESULT: <a href="http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-10-24&to=2012-10-24">report</a> |
Windows XP | PASS - 2012-10-24 |
Windows Vista | PASS - 2012-10-24 |
Windows 7 32-bit | PASS - 2012-10-24 |
Windows 7 64-bit | PASS - 2012-10-24 |
Windows 8 32-bit | PASS - 2012-10-24 |
Windows 8 64-bit | PASS - 2012-10-24 |
Mac OS X 10.6 | PASS - 2012-10-24 |
Mac OS X 10.7 | PASS - 2012-10-24 |
Mac OS X 10.8 | PASS - 2012-10-24 |
Ubuntu 12.04 32-bit | PASS - 2012-10-24 |
Ubuntu 12.04 64-bit | PASS - 2012-10-24 |
- Test Failures (<a href="https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA">known bugs</a>)
Manual Testing 8
There will be X bug verifications and testing around these bugs
- Results
Platform | Tester | 16.0.2 Candidates |
Windows XP | tester | RESULT - 2012-10-24 |
Windows Vista | tester | RESULT - 2012-10-24 |
Windows 7 32-bit | tester | RESULT - 2012-10-24 |
Windows 7 64-bit | tester | RESULT - 2012-10-24 |
Mac OS X 10.6 | tester | RESULT - 2012-10-24 |
Mac OS X 10.7 | Virgil | In progress - 2012-10-24 |
Mac OS X 10.8 | tester | RESULT - 2012-10-24 |
Ubuntu 12.04 32-bit | Ioana | In progress - 2012-10-25 |
Ubuntu 12.04 64-bit | Ioana | In progress - 2012-10-25 |
- Issues Found
Breakpad End-to-End 8
Test the end-to-end functionality in the crash reporting feature, across operating systems.
- Use the <a href="http://ted.mielczarek.org/mozilla/crashme.html">Crash Me add-on</a> to trigger a crash
- Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc
- Repeatedly crash Firefox and submit your reports
- Verify that your session restores properly
- Verify your crash reports are submitted properly by going through about:crashes and clicking the reports
- Try adding comments to your crash report and verify the crash-stats.com report shows the comment
Update Testing 8
- Partial: 14.0.1, 15.0.1, 16.0.1
- Complete: 12.0, 13.0
Platform | Releasetest: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-10-24&to=2012-10-25&target=16.0">report</a> | Beta: <a href="http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-10-24&to=2012-10-25&target=16.0">report</a> |
Windows XP | RESULT - 2012-10-24 | RESULT - 2012-10-24 |
Windows Vista | RESULT - 2012-10-24 | RESULT - 2012-10-24 |
Windows 7 32-bit | RESULT - 2012-10-24 | RESULT - 2012-10-24 |
Windows 7 64-bit | RESULT - 2012-10-24 | RESULT - 2012-10-24 |
Windows 8 32-bit | manual-see below - 2012-10-02 | Results - yyyy-mm-dd |
Windows 8 64-bit | manual-see below - 2012-10-02 | Results - yyyy-mm-dd |
Mac OS X 10.6 | RESULT - 2012-10-24 | RESULT - 2012-10-24 |
Mac OS X 10.7 | RESULT - 2012-10-24 | RESULT - 2012-10-24 |
Mac OS X 10.8 | RESULT - 2012-10-24 | RESULT - 2012-10-24 |
Ubuntu 12.04 32-bit | RESULT - 2012-10-24 | RESULT - 2012-10-24 |
Ubuntu 12.04 64-bit | RESULT - 2012-10-24 | RESULT - 2012-10-24 |
Windows 8: manual spotcheck until automation in place
- RELEASETEST
- RESULT - Firefox 16.0.1 en-US (partial)
- RESULT - Firefox 13.0 de (complete)
- RELEASE
- RESULT - Firefox 16.0.1 en-US (partial)
- RESULT - Firefox 12.0 fr (complete)
- Issues
Issues
Blockers
<thead> </thead> <tbody> </tbody>ID | Summary | Status | Priority |
---|---|---|---|
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=793126">793126</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=793126">crash with WebGL error message: "Failed to create temporary texture in system memory. Error code: 2147942414", while navigating street view on Google MapsGL</a> | NEW | -- |
Non-Blockers
<thead> </thead> <tbody> </tbody>ID | Summary | Status | Priority |
---|---|---|---|
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=786970">786970</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=786970">Windowed plugin content gets displayed out of it's container when scrolling the browser window</a> | NEW | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=786655">786655</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=786655">On flash content the context menu does not own focus</a> | NEW | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=786972">786972</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=786972">Can't replay the same HTML5 youtube video by clicking on the progress bar after when it ends</a> | NEW | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=787056">787056</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=787056">MapsGL maps.google.com renders blurry on machines Chrome doesn't allow MapsGL enabling on</a> | NEW | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=789463">789463</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=789463">[Vimeo] Cannot seek video using the scrubber when in couch mode</a> | NEW | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=797332">797332</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=797332">Characters typed in twitter's Compose new tweet text box aren't displayed after reaching the last row</a> | RESOLVED | -- |
Previously Known
<thead> </thead> <tbody> </tbody>ID | Summary | Status | Priority |
---|---|---|---|
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=779492">779492</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=779492">The youtube.com Flash pop out video player is opened in the background</a> | UNCONFIRMED | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=747909">747909</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=747909">In Full Screen mode - copy actions are not allowed on HTML5 videos</a> | NEW | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=675645">675645</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=675645">Firefox freezes when using Pop out while in Fullscreen mode on Youtube</a> | NEW | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=78414">78414</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=78414">Application shortcut keys (keyboard commands such as f11, ctrl+t, ctrl+r) fail to operate when plug-in (flash, acrobat, quicktime) has focus</a> | NEW | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=667399">667399</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=667399">RealPlayer G2 LiveConnect plugin crash</a> | NEW | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=768375">768375</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=768375">crash in nsDocShell::RestoreFromHistory</a> | NEW | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=775082">775082</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=775082">Video controls disappear after dragging the tabs</a> | NEW | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=781190">781190</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=781190">[adbe 3309208] Video images freeze with the latest Flash 11.4.400.252</a> | NEW | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=786655">786655</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=786655">On flash content the context menu does not own focus</a> | NEW | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=787056">787056</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=787056">MapsGL maps.google.com renders blurry on machines Chrome doesn't allow MapsGL enabling on</a> | NEW | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=791195">791195</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=791195">Session is not restored after Firefox crash</a> | NEW | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=792422">792422</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=792422">report-a-bug popup div on youtube - page jumping issue</a> | NEW | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=794924">794924</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=794924">Choppy fading out for Quicktime videos</a> | NEW | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=692446">692446</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=692446">Firefox prompts to install windows media player plugin even though it is already installed</a> | NEW | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=717178">717178</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=717178">Printing stops after 73%, and browser hangs up</a> | NEW | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=788423">788423</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=788423">[adbe 3325067] Flash hang in F_1152915508 on dropbox.com</a> | ASSIGNED | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=686782">686782</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=686782">GFX glitches on windows 8 (with D2D)</a> | REOPENED | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=785754">785754</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=785754">Buttons on some sites are not rendered right and are like half cut off</a> | RESOLVED | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=777051">777051</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=777051">Clicking "Replay" button on HTML5 Youtube Videos makes the video freeze at the beginning (and fire "ASSERTION: Seek target should lie inside the first audio block after seek [...] nsBuiltinDecoderStateMachine.cpp, line 1883")</a> | RESOLVED | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=778819">778819</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=778819">Toggle Fullscreen in context menu stops workng Flash 11.3.300.268</a> | RESOLVED | -- |
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=786237">786237</a> | <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=786237">Babylon toolbar is empty after installation</a> | RESOLVED | -- |