MozillaWiki

Releases/Firefox 16/Test Plan

< Releases
Revision as of 13:29, 20 July 2012 by VirgilD (talk | contribs) (→‎Manual Spot Checks and Feature Sign-Off)

« Firefox 15 | Firefox 16 | Firefox 17 »

Firefox 16 Test Plan

Summary

Milestone Firefox 16
Status Currently in Nightly -- merging to Aurora on July 17, 2012
Lead Juan Becerra (irc: juanb)
2nd Anthony Hughes (irc: ashughes)
Builds [ftp] [http] [web]

The following is the test plan for Firefox 16 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 16. After the release this document will be used as an archive of what was done to validate this release.

If you have some free time, please pick a task below and get in touch with one of the leads.

Details

The following is a description of the tasks which need to be completed throughout this release.

  • 2012-06-04: {{|}} Nightly Merge
  • 2012-07-17: {{|}} Aurora Merge
  • 2012-08-28: {{|}} Pre-Beta feature sign-offs
  • 2012-08-30: {{|}} Beta 1
  • 2012-09-06: {{|}} Beta 2
  • 2012-09-13: {{|}} Beta 3
  • 2012-09-20: {{|}} Beta 4
  • 2012-09-27: {{|}} Beta 5
  • 2012-10-04: {{|}} Beta 6
  • 2012-10-06: {{|}} Release Candidate
  • 2012-10-09: {{|}} Final

Nightly

Merge

Functional Automation - result
Update Spotchecks - result

Features

See the Feature Release Tracking wiki for a general overview

[DONE]
[ON TRACK]
[AT RISK]
[MISSED]

Endurance Automation

The purpose of this section is to track any potential memory regressions via spikes reported in the dashboard.

What to Do
  1. Check the Endurance Charts
  2. If you see an anomalous event (ie. spike or dip), get the report for the specific event
  3. If the event happens twice in a row, file a bug under Mozmill Tests
  4. Notify Anthony Hughes of the issue
  5. Record the event in this wiki below
Current Status (Mihaela Velimiroviciu) - Last Updated Jul 18th, 2012
Full Query
ID Summary Priority Status
764823 Memory spike discovered by endurance tests on Nightly Linux and Mac between June 12 and 13 -- RESOLVED
767348 Memory consumption shows an ascending trend on Nightly/Win 6.1 x64 -- RESOLVED
773632 High results on mozmill endurance test on Mac 10.7.4_x64 since Jul 5 -- RESOLVED

3 Total; 0 Open (0%); 3 Resolved (100%); 0 Verified (0%);


Functional Automation

The purpose of this section is to track any potential functional regressions via results reported in the dashboard.

What to Do
  1. Check the testrun reports
  2. If you see any failing tests, check that there are not any bugs on file for it already
  3. If not, file a bug under Mozmill Tests
  4. Notify Anthony Hughes of the issue
  5. Record the event in this wiki below
Current Status (Vlad Maniac / Alex Lakatos) - Last Updated


Bug Triage

The purpose of this section is to triage UNCONFIRMED incoming bugs.

What to Do
  1. Check the UNCONFIRMED bugs
  2. Test the bugs in the latest Firefox Nightly
  3. If you can't reproduce the bug, ask for the reporter to provide more information
  4. If you reproduce the bug, add your testing information and mark the bug NEW
  5. Notify Anthony Hughes of the NEW issue
Current Status (Vlad Ghetiu / Virgil Dicu) - Last Updated

Result Spreadsheet


Aurora Merge

The following are the detailed results of the automated BFT test runs.

Platform Result: report
Windows 2000 PASS - 2012-07-19
Windows XP PASS - 2012-07-19
Windows Vista PASS - 2012-07-19
Windows 7 32-bit PASS - 2012-07-19
Windows 7 64-bit PASS - 2012-07-19
Mac OS X 10.6 PASS - 2012-07-19
Mac OS X 10.7 PASS - 2012-07-19
Ubuntu 11.04 32-bit PASS - 2012-07-19
Ubuntu 11.04 64-bit PASS - 2012-07-19
Known Issues
Full Query
ID Summary Priority Status
719982 Failure in testAddons_RestartlessExtensionWorksAfterRestart | Modal dialog has been found and processed P2 RESOLVED
770117 Test failure "Number of visible rows should equal max rows - '1' should equal '6'" in /testAwesomeBar/testVisibleItemsMax.js P2 RESOLVED

2 Total; 0 Open (0%); 2 Resolved (100%); 0 Verified (0%);


Updates

The following are details of update verifications for Firefox 15 merge to Aurora

Auroratest
  • {{|}} [tester] Windows XP:
  • {{|}} [tester] Windows 7:
  • {{|}} [tester] Mac OS X 10.6:
  • {{|}} [tester] Ubuntu 11.04:
Aurora
  • {{|}} [tester] Windows XP:
  • {{|}} [tester] Windows 7:
  • {{|}} [tester] Mac OS X 10.6:
  • {{|}} [tester] Ubuntu 11.04:
Issues Found

Manual Spot Checks and Feature Sign-Off

Are all features implemented at this point working?

Manual spot Checks [spreadsheet]

  • Platforms: Windows XP, Windows 7 x64, Ubuntu 11.10, Mac 10.6
  • Found known issues:
  • New issues:
  • Status:

Run the Firefox 16 Aurora Merge Feature Sign-Off tests in MozTrap and report results below.

Feature Owner Win32 Mac Linux
Show PDF Inline Panel Based Download Manager Simona Badau Results - 2012-07-19 Results - 2012-07-19 Results - 2012-07-19
OS X 10.7 support Vlad Ghetiu N/A - 2012-07-19 N/A - 2012-07-19 N/A - 2012-07-19
Opt-in activation for plugins Paul Silaghi Results - 2012-07-19 Results - 2012-07-19 Results - 2012-07-19
Speedy Session Restore Paul Silaghi Results - 2012-07-19 Results - 2012-07-19 Results - 2012-07-19
Developer Toolbar Virgil Dicu PASS - 2012-07-19 PASS - 2012-07-19 PASS - 2012-07-19
Command Line Mihaela Velimiroviciu PASS - 2012-07-20 PASS - 2012-07-20 PASS - 2012-07-20
Show PDF Inline Mihaela Velimiroviciu PASS - 2012-07-20 PASS - 2012-07-20 PASS - 2012-07-20
Incremental Garbage Collection Ioana Budnar PASS - 2012-07-19 Results - 2012-07-19 Results - 2012-07-19

Notes:

Smoke tests

Run the smoketests in MozTrap and report results below.

Platform Assignee Result: report
Windows XP Simona PASS - 2012-07-19
Windows 7 64-bit Paul In progress - 2012-07-20
Mac OS X 10.7.4 VladG PASS - 2012-07-19
Ubuntu 12.04 32-bit Virgil PASS - 2012-07-19
Retrieved from "https://wiki.allizom.org/index.php?title=Releases/Firefox_16/Test_Plan&oldid=452819"