MozillaWiki

Releases/Firefox 16/Test Plan: Difference between revisions

Page Discussion

Releases/Firefox 16/Test Plan (view source)

Revision as of 09:57, 1 November 2012

5,270 bytes added ,  1 November 2012
→‎Previously Known
 
(25 intermediate revisions by 6 users not shown)
Line 2,401: Line 2,401:


; Checklist
; Checklist
* {{ok|}} [[Releases/Firefox_16/Test_Plan#Automated_Testing_7|Automated Testing]]
* {{done|}} [[Releases/Firefox_16/Test_Plan#Automated_Testing_7|Automated Testing]]
* {{ok|}} [[Releases/Firefox_16/Test_Plan#Manual_Testing_7|Manual Testing]]
* {{done|}} [[Releases/Firefox_16/Test_Plan#Manual_Testing_7|Manual Testing]]
* {{ok|}} [[Releases/Firefox_16/Test_Plan#Update_Testing_7|Update Testing]]
* {{done|}} [[Releases/Firefox_16/Test_Plan#Update_Testing_7|Update Testing]]
* {{ | }} Final sign-off sent and updates qualified on Beta channel
* {{ | }} Final sign-off sent and updates qualified on Release channel


=== Automated Testing 7 ===
=== Automated Testing 7 ===
Line 2,414: Line 2,414:
  |-
  |-
  | Windows XP
  | Windows XP
  | '''RESULT''' - 2012-10-10
  | '''PASS''' - 2012-10-10
  |-
  |-
  | Windows Vista
  | Windows Vista
  | '''RESULT''' - 2012-10-10
  | '''PASS''' - 2012-10-10
  |-
  |-
  | Windows 7 32-bit
  | Windows 7 32-bit
  | '''RESULT''' - 2012-10-10
  | '''PASS''' - 2012-10-10
  |-
  |-
  | Windows 7 64-bit
  | Windows 7 64-bit
  | '''RESULT''' - 2012-10-10
  | '''PASS''' - 2012-10-10
  |-
  |-
  | Windows 8 32-bit
  | Windows 8 32-bit
  | '''RESULT''' - 2012-10-10
  | '''PASS''' - 2012-10-10
  |-
  |-
  | Windows 8 64-bit
  | Windows 8 64-bit
  | '''RESULT''' - 2012-10-10
  | '''PASS''' - 2012-10-10
  |-
  |-
  | Mac OS X 10.6
  | Mac OS X 10.6
  | '''RESULT''' - 2012-10-10
  | '''PASS''' - 2012-10-10
  |-
  |-
  | Mac OS X 10.7
  | Mac OS X 10.7
  | '''RESULT''' - 2012-10-10
  | '''PASS''' - 2012-10-10
  |-
  |-
  | Mac OS X 10.8
  | Mac OS X 10.8
  | '''RESULT''' - 2012-10-10
  | '''PASS''' - 2012-10-10
  |-
  |-
  | Ubuntu 12.04 32-bit
  | Ubuntu 12.04 32-bit
  | '''RESULT''' - 2012-10-10
  | '''PASS''' - 2012-10-10
  |-
  |-
  | Ubuntu 12.04 64-bit
  | Ubuntu 12.04 64-bit
  | '''RESULT''' - 2012-10-10
  | '''PASS''' - 2012-10-10
  |}
  |}


Line 2,459: Line 2,459:
  |-
  |-
  | Windows XP
  | Windows XP
  | tester
  | juanb, ashughes, mwobensmith
  | '''RESULT''' - 2012-10-10
  | '''PASS''' - 2012-10-10
  |-
  |-
  | Windows Vista
  | Windows Vista
Line 2,471: Line 2,471:
  |-
  |-
  | Windows 7 64-bit
  | Windows 7 64-bit
  | tester
  | juanb, ashughes, mwobensmith
  | '''RESULT''' - 2012-10-10
  | '''PASS''' - 2012-10-10
  |-
  |-
  | Mac OS X 10.6
  | Mac OS X 10.6
Line 2,479: Line 2,479:
  |-
  |-
  | Mac OS X 10.7
  | Mac OS X 10.7
  | tester
  | juanb, ashughes, mwobensmith
  | '''RESULT''' - 2012-10-10
  | '''PASS''' - 2012-10-10
  |-
  |-
  | Mac OS X 10.8
  | Mac OS X 10.8
Line 2,491: Line 2,491:
  |-
  |-
  | Ubuntu 12.04 64-bit
  | Ubuntu 12.04 64-bit
  | tester
  | juanb, ashughes, mwobensmith
  | '''RESULT''' - 2012-10-10
  | '''PASS''' - 2012-10-10
  |}
  |}


Line 2,517: Line 2,517:
  |-
  |-
  | Windows XP
  | Windows XP
  | '''RESULT''' - 2012-10-10
  | '''PASS''' - 2012-10-11
  | '''RESULT''' - 2012-10-10
| '''PASS''' - 2012-10-11
|-
| Windows Vista
| '''PASS''' - 2012-10-11
| '''PASS''' - 2012-10-11
|-
| Windows 7 32-bit
| '''PASS''' - 2012-10-11
| '''PASS''' - 2012-10-11
|-
| Windows 7 64-bit
| '''PASS''' - 2012-10-11
| '''PASS''' - 2012-10-11
|-
| Windows 8 32-bit
| '''Pass'''- juanb - 2012-10-11
| '''Pass''' - juanb - 2012-10-11
|-
| Windows 8 64-bit
| '''Pass'''- juanb - 2012-10-11
| '''Pass''' - 2012-10-11
|-
| Mac OS X 10.6
| '''PASS''' - 2012-10-11
| '''PASS''' - 2012-10-11
|-
| Mac OS X 10.7
| '''PASS''' - 2012-10-11
| '''PASS''' - 2012-10-11
|-
| Mac OS X 10.8
| '''PASS''' - 2012-10-11
| '''PASS''' - 2012-10-11
|-
| Ubuntu 12.04 32-bit
| '''PASS''' - 2012-10-11
| '''PASS''' - 2012-10-11
|-
| Ubuntu 12.04 64-bit
| '''PASS''' - 2012-10-11
| '''PASS''' - 2012-10-11
|}
 
''Windows 8: manual spotcheck until automation in place''
* RELEASETEST
** '''PASS''' - Firefox 16.0 en-US (partial)
** '''PASS''' - Firefox 15.0.1 de (partial)
* RELEASE
** '''PASS''' - Firefox 16.0 en-US (partial)
** '''PASS''' - Firefox 12.0 fr (complete)
 
; Issues
 
== 16.0.2 ==
{| class="querytable" style="width:30%"
| style="width:10%" |
| style="width:20%" |
|-
| '''Builds'''
| [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0.2-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/16.0.2-candidates/build1/ HTTP]
|-
| '''ID'''
| 20121024073032
|-
| '''Changeset'''
| [https://hg.mozilla.org/releases/mozilla-release/rev/3995ff235342 3995ff235342] ([http://hg.mozilla.org/releases/mozilla-release/pushloghtml?fromchange=7789ab68a99d&tochange=3995ff235342 pushlog])
|}
 
; Checklist
* {{done|}} [[Releases/Firefox_16/Test_Plan#Automated_Testing_8|Automated Testing]]
* {{done|}} [[Releases/Firefox_16/Test_Plan#Manual_Testing_8|Manual Testing]]
* {{ok|}} [[Releases/Firefox_16/Test_Plan#Update_Testing_8|Update Testing]]
* {{ | }} Final sign-off sent and updates qualified on Beta channel
 
=== Automated Testing 8 ===
The following are the detailed results of the automated Mozmill test runs.
{|class="querytable" style="width:50%"
|-
| class="header" style="width:20%" | Platform
| class="header" style="width:30%" | RESULT: [http://mozmill-ci.blargon7.com/#/functional/reports?branch=16.0&platform=All&from=2012-10-24&to=2012-10-24 report]
|-
| Windows XP
| '''PASS''' - 2012-10-24
|-
| Windows Vista
| '''PASS''' - 2012-10-24
|-
| Windows 7 32-bit
| '''PASS''' - 2012-10-24
|-
| Windows 7 64-bit
| '''PASS''' - 2012-10-24
|-
| Windows 8 32-bit
| '''PASS''' - 2012-10-24
|-
| Windows 8 64-bit
| '''PASS''' - 2012-10-24
|-
| Mac OS X 10.6
| '''PASS''' - 2012-10-24
|-
| Mac OS X 10.7
| '''PASS''' - 2012-10-24
|-
| Mac OS X 10.8
| '''PASS''' - 2012-10-24
|-
| Ubuntu 12.04 32-bit
| '''PASS''' - 2012-10-24
|-
| Ubuntu 12.04 64-bit
| '''PASS''' - 2012-10-24
|}
 
; Test Failures ([https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA known bugs])
 
=== Manual Testing 8 ===
There will be X bug verifications and testing around these bugs. Basic functionality spot checking - [https://mozqa.etherpad.mozilla.org/Firefox16-0-2 etherpad].
 
; Results
{| class="querytable"
| class="header" | Platform
| class="header" | Tester
| class="header" | 16.0.2 Candidates
|-
| Windows XP
| Simona
| '''PASS''' - 2012-10-25
|-
| Windows Vista
| Simona
| '''PASS''' - 2012-10-25
|-
| Windows 7 32-bit
| Mihaela
| '''PASS''' - 2012-10-25
|-
| Windows 7 64-bit
| Manuela
| '''PASS''' - 2012-10-25
|-
| Mac OS X 10.6
| Virgil
| '''PASS''' - 2012-10-25
|-
| Mac OS X 10.7
| Virgil
| '''PASS''' - 2012-10-25
|-
| Mac OS X 10.8
| Mihaela
| '''PASS''' - 2012-10-25
|-
| Ubuntu 12.04 32-bit
| Ioana
| '''PASS''' - 2012-10-25
|-
| Ubuntu 12.04 64-bit
| Ioana
| '''PASS''' - 2012-10-25
|}
 
; Issues Found
 
==== Breakpad End-to-End 8 ====
Test the end-to-end functionality in the crash reporting feature, across operating systems.
* Use the [http://ted.mielczarek.org/mozilla/crashme.html Crash Me add-on] to trigger a crash
* Set up a dirty profile with lots of history, bookmarks, add-ons, themes, tabs, app-tabs, plugin content loaded, etc
* Repeatedly crash Firefox and submit your reports
* Verify that your session restores properly
* Verify your crash reports are submitted properly by going through about:crashes and clicking the reports
* Try adding comments to your crash report and verify the crash-stats.com report shows the comment
 
=== Update Testing 8 ===
* Partial: 14.0.1, 15.0.1, 16.0.1
* Complete: 12.0, 13.0
 
{|class="querytable" style="width:60%"
|-
| class="header" | Platform
| class="header" | Releasetest: [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=releasetest&from=2012-10-24&to=2012-10-25&target=16.0 report]
| class="header" | Beta: [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=16.0&channel=beta&from=2012-10-24&to=2012-10-25&target=16.0 report]
|-
| Windows XP
  | '''RESULT''' - 2012-10-24
| '''RESULT''' - 2012-10-24
  |-
  |-
  | Windows Vista
  | Windows Vista
  | '''RESULT''' - 2012-10-10
  | '''RESULT''' - 2012-10-24
  | '''RESULT''' - 2012-10-10
  | '''RESULT''' - 2012-10-24
  |-
  |-
  | Windows 7 32-bit
  | Windows 7 32-bit
  | '''RESULT''' - 2012-10-10
  | '''RESULT''' - 2012-10-24
  | '''RESULT''' - 2012-10-10
  | '''RESULT''' - 2012-10-24
  |-
  |-
  | Windows 7 64-bit
  | Windows 7 64-bit
  | '''RESULT''' - 2012-10-10
  | '''RESULT''' - 2012-10-24
  | '''RESULT''' - 2012-10-10
  | '''RESULT''' - 2012-10-24
  |-
  |-
  | Windows 8 32-bit
  | Windows 8 32-bit
  | '''manual'''-see below - 2012-10-10
  | '''manual'''-see below - 2012-10-02
  | '''Results''' - yyyy-mm-dd
  | '''Results''' - yyyy-mm-dd
  |-
  |-
  | Windows 8 64-bit
  | Windows 8 64-bit
  | '''manual'''-see below - 2012-10-10
  | '''manual'''-see below - 2012-10-02
  | '''Results''' - yyyy-mm-dd
  | '''Results''' - yyyy-mm-dd
  |-
  |-
  | Mac OS X 10.6
  | Mac OS X 10.6
  | '''RESULT''' - 2012-10-10
  | '''RESULT''' - 2012-10-24
  | '''RESULT''' - 2012-10-10
  | '''RESULT''' - 2012-10-24
  |-
  |-
  | Mac OS X 10.7
  | Mac OS X 10.7
  | '''RESULT''' - 2012-10-10
  | '''RESULT''' - 2012-10-24
  | '''RESULT''' - 2012-10-10
  | '''RESULT''' - 2012-10-24
  |-
  |-
  | Mac OS X 10.8
  | Mac OS X 10.8
  | '''RESULT''' - 2012-10-10
  | '''RESULT''' - 2012-10-24
  | '''RESULT''' - 2012-10-10
  | '''RESULT''' - 2012-10-24
  |-
  |-
  | Ubuntu 12.04 32-bit
  | Ubuntu 12.04 32-bit
  | '''RESULT''' - 2012-10-10
  | '''RESULT''' - 2012-10-24
  | '''RESULT''' - 2012-10-10
  | '''RESULT''' - 2012-10-24
  |-
  |-
  | Ubuntu 12.04 64-bit
  | Ubuntu 12.04 64-bit
  | '''RESULT''' - 2012-10-10
  | '''RESULT''' - 2012-10-24
  | '''RESULT''' - 2012-10-10
  | '''RESULT''' - 2012-10-24
  |}
  |}


''Windows 8: manual spotcheck until automation in place''
''Windows 8: manual spotcheck until automation in place''
* RELEASETEST
* RELEASETEST
** '''RESULT''' - Firefox 16.0 en-US (partial)
** '''RESULT''' - Firefox 16.0.1 en-US (partial)
** '''RESULT''' - Firefox 13.0 de (complete)
** '''RESULT''' - Firefox 13.0 de (complete)
* RELEASE
* RELEASE
** '''RESULT''' - Firefox 16.0 en-US (partial)
** '''RESULT''' - Firefox 16.0.1 en-US (partial)
** '''RESULT''' - Firefox 12.0 fr (complete)
** '''RESULT''' - Firefox 12.0 fr (complete)


Line 2,589: Line 2,775:
<bugzilla>
<bugzilla>
{
{
  "id": "768375, 686782, 667399, 785754, 779492, 775082, 777051, 747909, 787056, 675645, 778819, 786655, 781190, 78414, 788423, 786237, 791195, 717178, 794924, 692446, 792422"
  "id": "768375, 686782, 667399, 785754, 779492, 775082, 777051, 747909, 787056, 675645, 778819, 786655, 781190, 78414, 788423, 786237, 791195, 717178, 794924, 692446, 792422, 797116"
}
}
</bugzilla>
</bugzilla>
Simona Badau
Confirmed users
1,333

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/478578...484320"