Releases/Firefox 24/Test Plan: Difference between revisions
MihaiMorar (talk | contribs) |
MihaiMorar (talk | contribs) |
||
| Line 71: | Line 71: | ||
# Record the event in this wiki below | # Record the event in this wiki below | ||
; Current Status (Mihai Morar) - Last Updated - 2013. | ; Current Status (Mihai Morar) - Last Updated - 2013.07.26 | ||
<bugzilla> | <bugzilla> | ||
{ | { | ||
Revision as of 07:19, 7 August 2013
« Firefox 23 | Firefox 24 | Firefox 25 »
Firefox 24 Test Plan
Summary
| Milestone | Firefox 24 |
| Status | Currently in Beta - to Release on Week of September 17, 2013 |
| Lead | Tracy Walker (irc: tracy) |
| 2nd | Anthony Hughes (irc: ashughes) |
| Builds | [ftp] [http] [web] |
The following is the test plan for Firefox 24 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 24. After the release this document will be used as an archive of what was done to validate this release.
If you have some free time, please pick a task below and get in touch with one of the leads.
Details
The following is a description of the tasks which need to be completed throughout this release.
- 2013.05.14: [DONE] Nightly Merge
- 2013.06.24: [DONE] Aurora Merge
- 2013.08.02: [ON TRACK] Pre-Beta feature sign-offs
- 2013.08.08: [ON TRACK] Beta 1
- 2013.08.13: [ON TRACK] Beta 2
- 2013.08.15: [ON TRACK] Beta 3
- 2013.08.20: [ON TRACK] Beta 4
- 2013.08.22: [ON TRACK] Beta 5
- 2013.08.27: [ON TRACK] Beta 6
- 2013.08.29: [ON TRACK] Beta 7
- 2013.09.03: [ON TRACK] Beta 8
- 2013.09.05: [ON TRACK] Beta 9
- 2013.09.10: [ON TRACK] Beta 10
- 2013.09.13: [ON TRACK] Release Candidate
- 2013.09.17: [ON TRACK] Final
Nightly
Merge
- Functional Automation - result
- Update Spotchecks - result
Features
See the Feature Release Tracking wiki for a general overview
Features
See the Feature Release Tracking wiki for a general overview
| Feature | Owner | Status | Tests |
Endurance Automation
The purpose of this section is to track any potential memory regressions via spikes reported in the dashboard.
- What to Do
- Check the Endurance Charts
- If you see an anomalous event (ie. spike or dip), get the report for the specific event
- If the event happens twice in a row, file a bug under Mozmill Tests
- Notify Anthony Hughes of the issue
- Record the event in this wiki below
- Current Status (Mihai Morar) - Last Updated - 2013.07.26
| ID | Summary | Priority | Status |
|---|---|---|---|
| 887791 | Higher memory consumption on Mac 10.7.5 and 10.8.4 since June the 22nd | -- | VERIFIED |
1 Total; 0 Open (0%); 0 Resolved (0%); 1 Verified (100%);
Functional Automation
The purpose of this section is to track any potential functional regressions via results reported in the dashboard.
- What to Do
- Check the testrun reports
- If you see any failing tests, check that there are not any bugs on file for it already
- If not, file a bug under Mozmill Tests
- Notify Tracy Walker of the issue
- Record the event in this wiki below
- Current Status (SoftVision) - Last Updated - yyyy-mm-dd
Bug Triage
The purpose of this section is to triage UNCONFIRMED incoming bugs.
- What to Do
- Check the UNCONFIRMED bugs
- Test the bugs in the latest Firefox Nightly
- If you can't reproduce the bug, ask for the reporter to provide more information
- If you reproduce the bug, add your testing information and mark the bug NEW
- Notify Tracy Walker of the NEW issue
- Current Status (SoftVision) - Last Updated
Aurora Merge
| Builds | nightlies |
| Changeset | tip (pushlog) |
- Checklist
- [DONE] Automated Testing
- [DONE] Update Testing on auroratest
- {{|}} Update Testing on aurora
Automated Testing
The following are the detailed results of the automated Mozmill test runs.
| Platform | |
| Windows XP | [DONE] - 2013-06-27 |
| Windows Vista | [DONE] - 2013-06-27 |
| Windows 7 32-bit | [DONE] - 2013-06-27 |
| Windows 7 64-bit | [DONE] - 2013-06-27 |
| Windows 8 32-bit | [DONE] - 2013-06-27 |
| Windows 8 64-bit | [DONE] - 2013-06-27 |
| Mac OS X 10.6 | [DONE] - 2013-06-27 |
| Mac OS X 10.7 | [DONE] - 2013-06-27 |
| Mac OS X 10.8 | [DONE] - 2013-06-27 |
| Ubuntu 12.04 32-bit | [DONE] - 2013-06-27 |
| Ubuntu 12.04 64-bit | [DONE] - 2013-06-27 |
- Test Failures (known bugs)
Update Testing
Spotcheck updates from a previous Aurora to Aurora 24.
| Platform | auroratest channel | aurora channel |
| Windows 7 64-bit | [DONE]
en-US 23.a2 20130527004019 - 20130627004018 |
|
| Mac OS X 10.7 | [DONE]
22.a2 en-US 20130420004013 - 20130627004018 |
|
| Ubuntu 12.04 32-bit | [DONE]
en-US 21.a2 20130317042012 - 20130627004018 |
- Issues
Important Features/Bugs
Bug Triage
- VERIFYME: verify the fix and do some exploratory testing around the issue
- FIXED: triage these bugs and test to ensure the fix is effective
- QAWANTED: triage these bugs and test to fulfill the QA request
- UNCONFIRMED: triage these bugs for potential new regressions
QAWANTED
Bugs requiring QA's assistance in debugging (steps, regression range, exploratory testing, verification, etc)
Pre-beta Feature testing and Sign-off
Feature testing
- Platforms: Windows XP, Windows 7 x64, Ubuntu 11.10, Mac 10.7
Run the Firefox 24.0 Pre-beta Feature Sign off tests in MozTrap and report results below.
| Feature | Owner | Win32 | Mac | Linux |
Notes:
Smoke tests
Run the smoketests in MozTrap and report results below.
| Platform | Assignee | Result: |
| Windows XP | tester | RESULTS - yyyy-mm-dd |
| Windows 7 64-bit | tester | RESULTS - yyyy-mm-dd |
| Mac OS X 10.7.4 | tester | RESULTS - yyyy-mm-dd |
| Ubuntu 12.04 32-bit | tester | RESULTS - yyyy-mm-dd |
Beta
Beta 1
Build Info
| Builds | FTP, HTTP |
| Build ID | TBD |
| Changes | TDB |
| Configs | mozmill |
Checklist
| Task | Status |
| Feature Sign-off | [ON TRACK] |
| Regression Tests | [ON TRACK] |
| Automated Tests | [ON TRACK] |
| Bug Fix Verifications | [ON TRACK] |
| Update Tests (betatest) | [ON TRACK] |
| Update Tests (releasetest) | [ON TRACK] |
| Update Tests (beta) | [ON TRACK] |
| Stability & Support sign-off | [ON TRACK] |
Feature Sign Off
| Feature | QA Contact | Status |
| link to bug or feature page | contact(s) | [ON TRACK] status |
| Web Audio | Manuela Muntean | [DONE] |
Regression Tests
| Platform | Status (results) |
| Windows 7 32-bit | [ON TRACK] results |
| Windows 8 64-bit | [ON TRACK] results |
| Mac OSX 10.8 | [ON TRACK] results |
| Ubuntu Linux 32-bit | [ON TRACK] results |
Automated Tests
| Test | Result | Criteria |
| Automated Tests | results (details) | All Mozmill ondemand_functional testruns PASS with no new failures
|
Failures (known)
Bug Fix Verifications
50 Total; 0 Open (0%); 14 Resolved (28%); 36 Verified (72%);
Update Tests
| Test | Result | Criteria |
| Update Tests (betatest) |
result (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
| Update Tests (releasetest) |
result (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
| Update Tests (beta) |
result (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
New Issues
| Bugs | Status |
New Issues
Bugzilla query error
Query options must be valid JSON.1
Known Issues
Bugzilla query error
Query options must be valid JSON.1