QA/mozilla-aurora/fx5/Beta 4: Difference between revisions

From MozillaWiki
< QA‎ | mozilla-aurora‎ | fx5
Jump to navigation Jump to search
 
(24 intermediate revisions by 5 users not shown)
Line 21: Line 21:
=== Automation ===
=== Automation ===
Run automated BFTs and link reports from dashboard
Run automated BFTs and link reports from dashboard
 
* Reports: [https://litmus.mozilla.org/test_run_report.cgi?test_run_id=304 Litmus], [http://mozmill-release.brasstacks.mozilla.com/#/functional/reports?branch=5.0&platform=All&from=2011-06-03&to=2011-06-03 Mozmill]
{| class="fullwidth-table"
{| class="fullwidth-table"
  | style="background:#EFEFEF; width:20%" | '''Platform'''
  | style="background:#EFEFEF; width:20%" | '''Platform'''
Line 29: Line 29:
  |-
  |-
  | Windows 2000
  | Windows 2000
  |  
  | '''PASS'''
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/0f79b0c8982ff02807a3ce37627001f6 REPORT]
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/0f79b0c8982ff02807a3ce3762701001 REPORT]
  |-
  |-
  | Windows XP
  | Windows XP
  |  
  | '''PASS'''
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/0f79b0c8982ff02807a3ce3762701a97 REPORT]
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/0f79b0c8982ff02807a3ce37627043ec REPORT]
  |-
  |-
  | Windows Vista
  | Windows Vista
  |  
  | '''PASS'''
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/0f79b0c8982ff02807a3ce3762702e7f REPORT]
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/0f79b0c8982ff02807a3ce37627052b5 REPORT]
  |-
  |-
  | Windows 7
  | Windows 7
  |  
  | '''PASS'''
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/0f79b0c8982ff02807a3ce3762700de7 REPORT]
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/0f79b0c8982ff02807a3ce3762703ca7 REPORT]
  |-
  |-
  | Windows 7 64-bit
  | Windows 7 64-bit
  |  
  | '''PASS'''
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/0f79b0c8982ff02807a3ce3762702c02 REPORT]
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/0f79b0c8982ff02807a3ce3762706032 REPORT]
  |-
  |-
  | Mac OS X 10.6
  | Mac OS X 10.6
  |  
  | '''PASS'''
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/0f79b0c8982ff02807a3ce37627027b9 REPORT]
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/0f79b0c8982ff02807a3ce3762704e22 REPORT]
  |-
  |-
  | Ubuntu 10.10
  | Ubuntu 10.10
  |  
  | '''PASS'''
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/0f79b0c8982ff02807a3ce37626ff3db REPORT]
  | [ REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/0f79b0c8982ff02807a3ce3762700b40 REPORT]
  |-
  |-
  | Ubuntu 10.10 64-bit
  | Ubuntu 10.10 64-bit
  |  
  | '''PASS'''
  | [ REPORT] (i686)<br>[ REPORT] (x86_64)
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/0f79b0c8982ff02807a3ce376270698c REPORT] (i686)<br>[http://mozmill-release.brasstacks.mozilla.com/#/functional/report/0f79b0c8982ff02807a3ce3762700156 REPORT] (x86_64)
  | [ REPORT] (i686)<br>[ REPORT] (x86_64)
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/0f79b0c8982ff02807a3ce376270717d REPORT] (i686)<br>[http://mozmill-release.brasstacks.mozilla.com/#/functional/report/0f79b0c8982ff02807a3ce376270139c REPORT] (x86_64)
  |}
  |}


Line 83: Line 83:
|-
|-
| Windows 2000  
| Windows 2000  
|  
| George
|  
| PASS
|-
|-
| Windows XP  
| Windows XP  
|  
| VladG
|  
| PASS
|-
|-
| Windows Vista  
| Windows Vista  
|  
| Simona
|  
| PASS
|-
|-
| Windows 7  
| Windows 7  
|  
| AndreiD
|  
| PASS
|-
|-
| Windows 7 64-bit  
| Windows 7 64-bit  
|  
| AndreiD
|  
| PASS
|-
|-
| Mac OS X 10.6  
| Mac OS X 10.6  
|  
| VladG
|  
| PASS
|-
|-
| Ubuntu 10.10  
| Ubuntu 10.10  
|  
| Simona
|  
| PASS
|-
|-
| Ubuntu 10.10 64-bit  
| Ubuntu 10.10 64-bit  
|  
| Simona
|  
| PASS
|}
|}
;Issues Found
* [Win2000 only] {{Bug|662498}} - Breakpad dialog not displayed after crashing firefox with crashme extension


;BFTs
;BFTs
* WinXP  
* WinXP  
** Testers:  
** Testers: Andrei, Simona, Vlad
** Results: '''result''' in Litmus with known issues
** Results: '''PASS''' in Litmus with known issues


;Issues Found
;Issues Found
Line 127: Line 130:
{| class="fullwidth-table"
{| class="fullwidth-table"
  | style="background:#EFEFEF; width:25%" | '''Platform'''
  | style="background:#EFEFEF; width:25%" | '''Platform'''
  | style="background:#EFEFEF; width:25%" | '''Betatest''': [ report]
  | style="background:#EFEFEF; width:25%" | '''Betatest''': [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=5.0&channel=betatest&from=2011-06-06&to=2011-06-06&target=5.0 report]
  | style="background:#EFEFEF; width:25%" | '''Releasetest''': [ report]
  | style="background:#EFEFEF; width:25%" | '''Releasetest''': [ report]
  | style="background:#EFEFEF; width:25%" | '''Beta''': [ report]
  | style="background:#EFEFEF; width:25%" | '''Beta''': [ report]
  |-
  |-
  | Windows 2000
  | Windows 2000
  | en-US: '''result''' <br> de: '''result''' <br> es-ES: '''result''' <br> ru: '''result'''
  | en-US: '''PASS''' <br> de: '''PASS''' <br> es-ES: '''PASS''' <br> ru: '''PASS'''
  | en-US: '''result''' <br> de: '''result''' <br> es-ES: '''result''' <br> ru: '''result'''
  | en-US: '''result''' <br> de: '''result''' <br> es-ES: '''result''' <br> ru: '''result'''
  | en-US: '''result''' <br> de: '''result''' <br> es-ES: '''result''' <br> ru: '''result'''
  | en-US: '''result''' <br> de: '''result''' <br> es-ES: '''result''' <br> ru: '''result'''
  |-
  |-
  | Windows XP
  | Windows XP
  | en-US: '''result''' <br> es-ES: '''result''' <br> fr: '''result''' <br> ru: '''result'''
  | en-US: '''PASS''' <br> es-ES: '''PASS''' <br> fr: '''PASS''' <br> ru: '''PASS'''
  | en-US: '''result''' <br> es-ES: '''result''' <br> fr: '''result''' <br> ru: '''result'''
  | en-US: '''result''' <br> es-ES: '''result''' <br> fr: '''result''' <br> ru: '''result'''
  | en-US: '''result''' <br> es-ES: '''result''' <br> fr: '''result''' <br> ru: '''result'''
  | en-US: '''result''' <br> es-ES: '''result''' <br> fr: '''result''' <br> ru: '''result'''
  |-
  |-
  | Windows Vista
  | Windows Vista
  | en-US: '''result''' <br> es-ES: '''result''' <br> fr: '''result''' <br> ru: '''result'''
  | en-US: '''PASS''' <br> es-ES: '''PASS''' <br> fr: '''PASS''' <br> ru: '''PASS'''
  | en-US: '''result''' <br> es-ES: '''result''' <br> fr: '''result''' <br> ru: '''result'''
  | en-US: '''result''' <br> es-ES: '''result''' <br> fr: '''result''' <br> ru: '''result'''
  | en-US: '''result''' <br> es-ES: '''result''' <br> fr: '''result''' <br> ru: '''result'''
  | en-US: '''result''' <br> es-ES: '''result''' <br> fr: '''result''' <br> ru: '''result'''
  |-
  |-
  | Windows 7
  | Windows 7
  | en-US: '''result''' <br> de: '''result''' <br> ru: '''result''' <br> zh-TW: '''result'''
  | en-US: '''PASS''' <br> de: '''PASS''' <br> ru: '''PASS''' <br> zh-TW: '''PASS'''
  | en-US: '''result''' <br> de: '''result''' <br> ru: '''result''' <br> zh-TW: '''result'''
  | en-US: '''result''' <br> de: '''result''' <br> ru: '''result''' <br> zh-TW: '''result'''
  | en-US: '''result''' <br> de: '''result''' <br> ru: '''result''' <br> zh-TW: '''result'''
  | en-US: '''result''' <br> de: '''result''' <br> ru: '''result''' <br> zh-TW: '''result'''
  |-
  |-
  | Windows 7 64-bit
  | Windows 7 64-bit
  | en-US: '''result''' <br> de: '''result''' <br> ru: '''result''' <br> zh-TW: '''result'''
  | en-US: '''PASS''' <br> de: '''PASS''' <br> ru: '''PASS''' <br> zh-TW: '''PASS'''
  | en-US: '''result''' <br> de: '''result''' <br> ru: '''result''' <br> zh-TW: '''result'''
  | en-US: '''result''' <br> de: '''result''' <br> ru: '''result''' <br> zh-TW: '''result'''
  | en-US: '''result''' <br> de: '''result''' <br> ru: '''result''' <br> zh-TW: '''result'''
  | en-US: '''result''' <br> de: '''result''' <br> ru: '''result''' <br> zh-TW: '''result'''
  |-
  |-
  | Mac OS X 10.6
  | Mac OS X 10.6
  | en-US: '''result''' <br> de: '''result''' <br> fr: '''result'''
  | en-US: '''PASS''' <br> de: '''PASS''' <br> fr: '''PASS'''
  | en-US: '''result''' <br> de: '''result''' <br> fr: '''result'''
  | en-US: '''result''' <br> de: '''result''' <br> fr: '''result'''
  | en-US: '''result''' <br> de: '''result''' <br> fr: '''result'''
  | en-US: '''result''' <br> de: '''result''' <br> fr: '''result'''
  |-
  |-
  | Ubuntu 10.10
  | Ubuntu 10.10
  | en-US: '''result''' <br> de: '''result''' <br> es-ES: '''result''' <br> fr: '''result''' <br> zh-TW: '''result'''
  | en-US: '''PASS''' <br> de: '''PASS''' <br> es-ES: '''PASS''' <br> fr: '''PASS''' <br> zh-TW: '''PASS'''
  | en-US: '''result''' <br> de: '''result''' <br> es-ES: '''result''' <br> fr: '''result''' <br> zh-TW: '''result'''
  | en-US: '''result''' <br> de: '''result''' <br> es-ES: '''result''' <br> fr: '''result''' <br> zh-TW: '''result'''
  | en-US: '''result''' <br> de: '''result''' <br> es-ES: '''result''' <br> fr: '''result''' <br> zh-TW: '''result'''
  | en-US: '''result''' <br> de: '''result''' <br> es-ES: '''result''' <br> fr: '''result''' <br> zh-TW: '''result'''
  |-
  |-
  | Ubuntu 10.10 64-bit
  | Ubuntu 10.10 64-bit
  | en-US: '''result''' <br> de: '''result''' <br> es-ES: '''result''' <br> fr: '''result''' <br> zh-TW: '''result'''
  | en-US: '''PASS''' <br> de: '''PASS''' <br> es-ES: '''PASS''' <br> fr: '''PASS''' <br> zh-TW: '''PASS'''
  | en-US: '''result''' <br> de: '''result''' <br> es-ES: '''result''' <br> fr: '''result''' <br> zh-TW: '''result'''
  | en-US: '''result''' <br> de: '''result''' <br> es-ES: '''result''' <br> fr: '''result''' <br> zh-TW: '''result'''
  | en-US: '''result''' <br> de: '''result''' <br> es-ES: '''result''' <br> fr: '''result''' <br> zh-TW: '''result'''
  | en-US: '''result''' <br> de: '''result''' <br> es-ES: '''result''' <br> fr: '''result''' <br> zh-TW: '''result'''
Line 194: Line 197:
  |  
  |  
  |}
  |}
; Known Failures
* Sporadic automation failures due to slow download times in conflict with timeout values in tests -- failed locales spotchecked ok.
* Desktop link/folder does not update to "beta" with channel switcher


; Issues Found
; Issues Found
* n/a
* Spot-checking on Mac OS X 10.6, Ubuntu 11.04, WinXP and Win 7 - no issues found


== Problems Found ==
== Problems Found ==
List any found issues or bugs here. Use '''[5b4]''' whiteboard entry to keep track.
List any found issues or bugs here. Use '''[5b4]''' whiteboard entry to keep track.

Latest revision as of 14:51, 7 June 2011

« mozilla-aurora | mozilla-aurora:fx5 | mozilla-aurora:fx6 »

Beta 4 Test Results

The following spotchecks should be performed before signing off on the next weekly beta.

  • [ON TRACK] Manual Testing - result
  • [ON TRACK] Automation (Mozmill BFTs) - result
  • [ON TRACK] Automation (Updates) - result
  • [ON TRACK] Stability Report (crash-stats) - result
  • [ON TRACK] Bug Verifications - result

Platform Coverage

All testing/spotchecks should be performed on the following operating systems:

  • Windows 7 (32-bit & 64-bit)
  • Windows Vista
  • Windows XP
  • Windows 2000
  • Mac OSX 10.6
  • Ubuntu 10.10 (32-bit and 64-bit)

Results

Automation

Run automated BFTs and link reports from dashboard

Platform Result General Tests Restart Tests
Windows 2000 PASS REPORT REPORT
Windows XP PASS REPORT REPORT
Windows Vista PASS REPORT REPORT
Windows 7 PASS REPORT REPORT
Windows 7 64-bit PASS REPORT REPORT
Mac OS X 10.6 PASS REPORT REPORT
Ubuntu 10.10 PASS REPORT REPORT
Ubuntu 10.10 64-bit PASS REPORT (i686)
REPORT (x86_64) 		REPORT (i686)
REPORT (x86_64)
Issues Found
  • n/a

Manual

Smoketests

Litmus Testrun: https://litmus.mozilla.org/run_tests.cgi?test_run_id=304

Platform Name of tester Results
Windows 2000 George PASS
Windows XP VladG PASS
Windows Vista Simona PASS
Windows 7 AndreiD PASS
Windows 7 64-bit AndreiD PASS
Mac OS X 10.6 VladG PASS
Ubuntu 10.10 Simona PASS
Ubuntu 10.10 64-bit Simona PASS
Issues Found
  • [Win2000 only] bug 662498 - Breakpad dialog not displayed after crashing firefox with crashme extension
BFTs
  • WinXP
    • Testers: Andrei, Simona, Vlad
    • Results: PASS in Litmus with known issues
Issues Found
  • n/a

Updates

Mozmill Results
Platform Betatest: report Releasetest: [ report] Beta: [ report]
Windows 2000 en-US: PASS
de: PASS
es-ES: PASS
ru: PASS 		en-US: result
de: result
es-ES: result
ru: result 		en-US: result
de: result
es-ES: result
ru: result
Windows XP en-US: PASS
es-ES: PASS
fr: PASS
ru: PASS 		en-US: result
es-ES: result
fr: result
ru: result 		en-US: result
es-ES: result
fr: result
ru: result
Windows Vista en-US: PASS
es-ES: PASS
fr: PASS
ru: PASS 		en-US: result
es-ES: result
fr: result
ru: result 		en-US: result
es-ES: result
fr: result
ru: result
Windows 7 en-US: PASS
de: PASS
ru: PASS
zh-TW: PASS 		en-US: result
de: result
ru: result
zh-TW: result 		en-US: result
de: result
ru: result
zh-TW: result
Windows 7 64-bit en-US: PASS
de: PASS
ru: PASS
zh-TW: PASS 		en-US: result
de: result
ru: result
zh-TW: result 		en-US: result
de: result
ru: result
zh-TW: result
Mac OS X 10.6 en-US: PASS
de: PASS
fr: PASS 		en-US: result
de: result
fr: result 		en-US: result
de: result
fr: result
Ubuntu 10.10 en-US: PASS
de: PASS
es-ES: PASS
fr: PASS
zh-TW: PASS 		en-US: result
de: result
es-ES: result
fr: result
zh-TW: result 		en-US: result
de: result
es-ES: result
fr: result
zh-TW: result
Ubuntu 10.10 64-bit en-US: PASS
de: PASS
es-ES: PASS
fr: PASS
zh-TW: PASS 		en-US: result
de: result
es-ES: result
fr: result
zh-TW: result 		en-US: result
de: result
es-ES: result
fr: result
zh-TW: result
Channel Switching Spotchecks
Platform Build IDs Result
Windows XP
Windows 7
Mac OS X 10.6
Ubuntu 10.10
Issues Found
  • Spot-checking on Mac OS X 10.6, Ubuntu 11.04, WinXP and Win 7 - no issues found

Problems Found

List any found issues or bugs here. Use [5b4] whiteboard entry to keep track.

Retrieved from "https://wiki.allizom.org/index.php?title=QA/mozilla-aurora/fx5/Beta_4&oldid=316157"