QA/mozilla-aurora/fx5/Beta 3: Difference between revisions

From MozillaWiki
< QA‎ | mozilla-aurora‎ | fx5
Jump to navigation Jump to search
No edit summary
 
(21 intermediate revisions by 6 users not shown)
Line 3: Line 3:
== Beta 3 Test Results ==
== Beta 3 Test Results ==
The following spotchecks should be performed before signing off on the next weekly beta.
The following spotchecks should be performed before signing off on the next weekly beta.
* {{ok|'''In Progress'''}} Manual Testing - '''[[QA/mozilla-aurora/fx5/Beta_3/Manual_Testing|Manual Testing]]'''
* {{done|'''Signed-Off'''}} Manual Testing - '''[[QA/mozilla-aurora/fx5/Beta_3/Manual_Testing|Manual Testing]]'''
* {{ok|'''In Progress'''}} Automation (Mozmill BFTs) - [[QA/mozilla-aurora/fx5/Beta_3#Automation|'''Results''']]
* {{done|}} Automation (Mozmill BFTs) - [[QA/mozilla-aurora/fx5/Beta_3#Automation|'''PASS''']]
* {{ok|'''In Progress'''}} [[QA/mozilla-aurora/fx5/Beta_3/Updates_Testing|Updates Test Automation]] (Mozimill)
* {{done|}} Automation (Updates) - [[QA/mozilla-aurora/fx5/Beta_3#Updates|'''PASS''']] - betatest
* {{ok|'''In Progress'''}} Stability Report (crash-stats)
* {{done|'''Signed-Off'''}} Stability Report (crash-stats)
* {{ok|'''In Progress'''}} Bug Verifications
** No obvious top crash regressions
* {{ok|'''Signed-Off'''}} Bug Verifications
** Focused on bug verifications with high user impact, and those that could be verified.


== Platform Coverage ==
== Platform Coverage ==
Line 29: Line 31:
  |-
  |-
  | Windows 2000
  | Windows 2000
  |  
  | '''PASS'''
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/d5e6504a1bcd108073bcb3e1df2c992d REPORT]
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/d5e6504a1bcd108073bcb3e1df2cd101 REPORT]
  |-
  |-
  | Windows XP
  | Windows XP
  |  
  | '''PASS'''
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/d5e6504a1bcd108073bcb3e1df2cb159 REPORT]
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/d5e6504a1bcd108073bcb3e1df2cf705 REPORT]
  |-
  |-
  | Windows Vista
  | Windows Vista
  |  
  | '''PASS'''
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/d5e6504a1bcd108073bcb3e1df2cb68e REPORT]
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/d5e6504a1bcd108073bcb3e1df2d00ba REPORT]
  |-
  |-
  | Windows 7
  | Windows 7
  |  
  | '''PASS'''
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/d5e6504a1bcd108073bcb3e1df2ca868 REPORT]
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/d5e6504a1bcd108073bcb3e1df2d0097 REPORT]
  |-
  |-
  | Windows 7 64-bit
  | Windows 7 64-bit
  |  
  | '''PASS'''
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/d5e6504a1bcd108073bcb3e1df2cdd4f REPORT]
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/d5e6504a1bcd108073bcb3e1df2d1822 REPORT]
  |-
  |-
  | Mac OS X 10.6
  | Mac OS X 10.6
  |  
  | '''PASS'''
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/d5e6504a1bcd108073bcb3e1df2cc4df REPORT]
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/d5e6504a1bcd108073bcb3e1df2d0e6a REPORT]
  |-
  |-
  | Ubuntu 10.10
  | Ubuntu 10.10
  |  
  | '''PASS'''
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/d5e6504a1bcd108073bcb3e1df2c7ef8 REPORT]
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/d5e6504a1bcd108073bcb3e1df2ced07 REPORT]
  |-
  |-
  | Ubuntu 10.10 64-bit
  | Ubuntu 10.10 64-bit
  |  
  | '''PASS'''
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/d5e6504a1bcd108073bcb3e1df2ce998 REPORT] (i686)<br>[http://mozmill-release.brasstacks.mozilla.com/#/general/report/d5e6504a1bcd108073bcb3e1df2d2ced REPORT] (x86_64)
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/d5e6504a1bcd108073bcb3e1df2ced07 REPORT] (i686)<br>[http://mozmill-release.brasstacks.mozilla.com/#/general/report/d5e6504a1bcd108073bcb3e1df2d399e REPORT] (x86_64)
  |}
  |}


; Issues Found:
; Issues Found:
* n/a
=== Manual  ===
;Smoketests
:
{| class="fullwidth-table"
|-
| style="background:#EFEFEF; width:20%" | '''Platform'''
| style="background:#EFEFEF; width:25%" | '''Name of tester'''
| style="background:#EFEFEF; width:25%" | '''Results'''
|-
| Windows 2000
| AndreiD
| '''PASS'''
|-
| Windows XP
| Vlad Ghetiu
| '''PASS'''
|-
| Windows Vista
| Simona
| '''PASS'''
|-
| Windows 7
| AndreiD
| '''PASS'''
|-
| Windows 7 64-bit
| AndreiD
| '''PASS'''
|-
| Mac OS X 10.6
| Vlad Ghetiu
| '''PASS'''
|-
| Ubuntu 10.10
| George
| '''PASS'''
|-
| Ubuntu 10.10 64-bit
| Simona
| '''PASS'''
|}
;BFTs
:
*WinXP
*Testers: Andrei, George, Simona, Vlad
*Results: '''PASS''' in Litmus with known issues
;Issues Found
:
*n/a
=== Updates ===
; Mozmill Results
{| class="fullwidth-table"
| style="background:#EFEFEF; width:25%" | '''Platform'''
| style="background:#EFEFEF; width:25%" | '''Betatest''': [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=5.0&channel=betatest&from=2011-05-24&to=2011-05-31&target=5.0 report]
| style="background:#EFEFEF; width:25%" | '''Releasetest''': [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=5.0&channel=releasetest&from=2011-05-25&to=2011-06-01&target=5.0 report]
| style="background:#EFEFEF; width:25%" | '''Beta''': [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=5.0&channel=beta&from=2011-05-25&to=2011-06-01&target=5.0 report]
|-
| Windows 2000
| en-US: '''PASS''' <br> de: '''PASS''' <br> es-ES: '''PASS''' <br> ru: '''PASS'''
| en-US: '''PASS''' <br> de: '''PASS''' <br> es-ES: '''PASS''' <br> ru: '''PASS'''
| en-US: '''PASS''' <br> de: '''PASS''' <br> es-ES: '''PASS''' <br> ru: '''PASS'''
|-
| Windows XP
| en-US: '''PASS''' <br> es-ES: '''PASS''' <br> fr: '''PASS''' <br> ru: '''PASS'''
| en-US: '''PASS''' <br> es-ES: '''PASS''' <br> fr: '''PASS''' <br> ru: '''PASS'''
| en-US: '''PASS''' <br> es-ES: '''PASS''' <br> fr: '''PASS''' <br> ru: '''PASS'''
|-
| Windows Vista
| en-US: '''PASS''' <br> es-ES: '''PASS''' <br> fr: '''PASS''' <br> ru: '''PASS'''
| en-US: '''PASS''' <br> es-ES: '''PASS''' <br> fr: '''PASS''' <br> ru: '''PASS'''
| en-US: '''PASS''' <br> es-ES: '''PASS''' <br> fr: '''PASS''' <br> ru: '''PASS'''
|-
| Windows 7
| en-US: '''PASS''' <br> de: '''PASS''' <br> ru: '''PASS''' <br> zh-TW: '''PASS'''
| en-US: '''PASS''' <br> de: '''PASS''' <br> ru: '''PASS''' <br> zh-TW: '''PASS'''
| en-US: '''PASS''' <br> de: '''PASS''' <br> ru: '''PASS''' <br> zh-TW: '''PASS'''
|-
| Windows 7 64-bit
| en-US: '''PASS''' <br> de: '''PASS''' <br> ru: '''PASS''' <br> zh-TW: '''PASS'''
| en-US: '''PASS''' <br> de: '''PASS''' <br> ru: '''PASS''' <br> zh-TW: '''PASS'''
| en-US: '''PASS''' <br> de: '''PASS''' <br> ru: '''PASS''' <br> zh-TW: '''PASS'''
|-
| Mac OS X 10.6
| en-US: '''PASS''' <br> de: '''PASS''' <br> fr: '''PASS'''
| en-US: '''PASS''' <br> de: '''PASS''' <br> fr: '''PASS'''
| en-US: '''PASS''' <br> de: '''PASS''' <br> fr: '''PASS'''
|-
| Ubuntu 10.10
| en-US: '''PASS''' <br> de: '''PASS''' <br> es-ES: '''PASS''' <br> fr: '''PASS''' <br> zh-TW: '''PASS'''
| en-US: '''PASS''' <br> de: '''PASS''' <br> es-ES: '''PASS''' <br> fr: '''PASS''' <br> zh-TW: '''PASS'''
| en-US: '''PASS''' <br> de: '''PASS''' <br> es-ES: '''PASS''' <br> fr: '''PASS''' <br> zh-TW: '''PASS'''
|-
| Ubuntu 10.10 64-bit
| en-US: '''PASS''' <br> de: '''PASS''' <br> es-ES: '''PASS''' <br> fr: '''PASS''' <br> zh-TW: '''PASS'''
| en-US: '''PASS''' <br> de: '''PASS''' <br> es-ES: '''PASS''' <br> fr: '''PASS''' <br> zh-TW: '''PASS'''
| en-US: '''PASS''' <br> de: '''PASS''' <br> es-ES: '''PASS''' <br> fr: '''PASS''' <br> zh-TW: '''PASS'''
|}
; Channel Switching Spotchecks
{| class="fullwidth-table"
| style="background:#EFEFEF; width:25%" | '''Platform'''
| style="background:#EFEFEF; width:65%" | '''Build IDs'''
| style="background:#EFEFEF; width:10%" | '''Result'''
|-
| Windows XP
| Firefox 6.0a2 20110601 -> Firefox 5.0b3 20110527
| '''PASS'''
|-
| Windows 7
| Firefox 6.0a2 20110601 -> Firefox 5.0b3 20110527
| '''PASS'''
|-
| Mac OS X 10.6
| Firefox 6.0a2 20110601 -> Firefox 5.0b3 20110527
| '''PASS'''
|-
| Ubuntu 10.10
| Firefox 6.0a2 20110601 -> Firefox 5.0b3 20110527
| '''PASS'''
|}
; Known Failures
* Sporadic automation failures due to slow download times in conflict with timeout values in tests -- failed locales spotchecked ok.
* Desktop link/folder does not update to "beta" with channel switcher
; Issues Found
* n/a
* n/a


== Problems Found ==
== Problems Found ==
List any found issues or bugs here. Use '''[5b3]''' whiteboard entry to keep track.
List any found issues or bugs here. Use '''[5b3]''' whiteboard entry to keep track.

Latest revision as of 21:12, 1 June 2011

« mozilla-aurora | mozilla-aurora:fx5 | mozilla-aurora:fx6 »

Beta 3 Test Results

The following spotchecks should be performed before signing off on the next weekly beta.

  • [DONE] Signed-Off Manual Testing - Manual Testing
  • [DONE] Automation (Mozmill BFTs) - PASS
  • [DONE] Automation (Updates) - PASS - betatest
  • [DONE] Signed-Off Stability Report (crash-stats)
    • No obvious top crash regressions
  • [ON TRACK] Signed-Off Bug Verifications
    • Focused on bug verifications with high user impact, and those that could be verified.

Platform Coverage

All testing/spotchecks should be performed on the following operating systems:

  • Windows 7 (32-bit & 64-bit)
  • Windows Vista
  • Windows XP
  • Windows 2000
  • Mac OSX 10.6
  • Ubuntu 10.10 (32-bit and 64-bit)

Results

Automation

Run automated BFTs and link reports from dashboard

Platform Result General Tests Restart Tests
Windows 2000 PASS REPORT REPORT
Windows XP PASS REPORT REPORT
Windows Vista PASS REPORT REPORT
Windows 7 PASS REPORT REPORT
Windows 7 64-bit PASS REPORT REPORT
Mac OS X 10.6 PASS REPORT REPORT
Ubuntu 10.10 PASS REPORT REPORT
Ubuntu 10.10 64-bit PASS REPORT (i686)
REPORT (x86_64) 		REPORT (i686)
REPORT (x86_64)
Issues Found
  • n/a

Manual

Smoketests
Platform Name of tester Results
Windows 2000 AndreiD PASS
Windows XP Vlad Ghetiu PASS
Windows Vista Simona PASS
Windows 7 AndreiD PASS
Windows 7 64-bit AndreiD PASS
Mac OS X 10.6 Vlad Ghetiu PASS
Ubuntu 10.10 George PASS
Ubuntu 10.10 64-bit Simona PASS
BFTs
  • WinXP
  • Testers: Andrei, George, Simona, Vlad
  • Results: PASS in Litmus with known issues
Issues Found
  • n/a

Updates

Mozmill Results
Platform Betatest: report Releasetest: report Beta: report
Windows 2000 en-US: PASS
de: PASS
es-ES: PASS
ru: PASS 		en-US: PASS
de: PASS
es-ES: PASS
ru: PASS 		en-US: PASS
de: PASS
es-ES: PASS
ru: PASS
Windows XP en-US: PASS
es-ES: PASS
fr: PASS
ru: PASS 		en-US: PASS
es-ES: PASS
fr: PASS
ru: PASS 		en-US: PASS
es-ES: PASS
fr: PASS
ru: PASS
Windows Vista en-US: PASS
es-ES: PASS
fr: PASS
ru: PASS 		en-US: PASS
es-ES: PASS
fr: PASS
ru: PASS 		en-US: PASS
es-ES: PASS
fr: PASS
ru: PASS
Windows 7 en-US: PASS
de: PASS
ru: PASS
zh-TW: PASS 		en-US: PASS
de: PASS
ru: PASS
zh-TW: PASS 		en-US: PASS
de: PASS
ru: PASS
zh-TW: PASS
Windows 7 64-bit en-US: PASS
de: PASS
ru: PASS
zh-TW: PASS 		en-US: PASS
de: PASS
ru: PASS
zh-TW: PASS 		en-US: PASS
de: PASS
ru: PASS
zh-TW: PASS
Mac OS X 10.6 en-US: PASS
de: PASS
fr: PASS 		en-US: PASS
de: PASS
fr: PASS 		en-US: PASS
de: PASS
fr: PASS
Ubuntu 10.10 en-US: PASS
de: PASS
es-ES: PASS
fr: PASS
zh-TW: PASS 		en-US: PASS
de: PASS
es-ES: PASS
fr: PASS
zh-TW: PASS 		en-US: PASS
de: PASS
es-ES: PASS
fr: PASS
zh-TW: PASS
Ubuntu 10.10 64-bit en-US: PASS
de: PASS
es-ES: PASS
fr: PASS
zh-TW: PASS 		en-US: PASS
de: PASS
es-ES: PASS
fr: PASS
zh-TW: PASS 		en-US: PASS
de: PASS
es-ES: PASS
fr: PASS
zh-TW: PASS
Channel Switching Spotchecks
Platform Build IDs Result
Windows XP Firefox 6.0a2 20110601 -> Firefox 5.0b3 20110527 PASS
Windows 7 Firefox 6.0a2 20110601 -> Firefox 5.0b3 20110527 PASS
Mac OS X 10.6 Firefox 6.0a2 20110601 -> Firefox 5.0b3 20110527 PASS
Ubuntu 10.10 Firefox 6.0a2 20110601 -> Firefox 5.0b3 20110527 PASS
Known Failures
  • Sporadic automation failures due to slow download times in conflict with timeout values in tests -- failed locales spotchecked ok.
  • Desktop link/folder does not update to "beta" with channel switcher
Issues Found
  • n/a

Problems Found

List any found issues or bugs here. Use [5b3] whiteboard entry to keep track.

Retrieved from "https://wiki.allizom.org/index.php?title=QA/mozilla-aurora/fx5/Beta_3&oldid=314785"