QA/Firefox3.5/TestPlan/BetaPreview: Difference between revisions

From MozillaWiki
< QA‎ | Firefox3.5‎ | TestPlan
Jump to navigation Jump to search
 
(34 intermediate revisions by 7 users not shown)
Line 21: Line 21:
* See Test Coverage below
* See Test Coverage below


== Build Location and IDs ==
== Build Location and IDs (Build 2) ==
Download:  
Download:  
* Mac->http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/3.5b99-candidates/build2/mac/en-US/
* Linux: ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/3.5b99-candidates/build2/linux-i686/


Builds:
Builds:
* Intel Mac
**Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b99) Gecko/20090605144508 Firefox/3.5b99
**about:buildconfig -> http://hg.mozilla.org/releases/mozilla-1.9.1/rev/3f775c3bb24a
* Linux
** Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1b99) Gecko/20090605 Firefox/3.5b99
** about:buildconfig: http://hg.mozilla.org/releases/mozilla-1.9.1/rev/3f775c3bb24a


== Test Coverage ==  
== Test Coverage ==  
Line 46: Line 58:
** Bug Fixes
** Bug Fixes


== Test Results ==
== Test Results Build2==
{| class="fullwidth-table"
{| class="fullwidth-table"
! Types of Tests
! Types of Tests
Line 54: Line 66:
|-
|-
| Smoketests
| Smoketests
| tracy - '''result'''  
| tracy - '''PASS''' ''with noted migration bug''
| hskupin - '''result'''
| marci/juanb/hskupin - '''PASS''' ''pending result on ppc machine''
| juanb/ashughes - '''result'''   
| juanb - '''PASS'''   
|-
|-
| L10n
| L10n
| tchung - '''result'''
| tchung - '''PASS'''
| tchung - '''PASS'''
| tchung - '''PASS'''
| tchung - '''PASS'''
| tchung - '''PASS'''
Line 65: Line 77:
| Updates
| Updates
| tracy<BR> '''Betatest - result''' <BR> '''releasetest - result''' <BR> '''Beta - result'''  
| tracy<BR> '''Betatest - result''' <BR> '''releasetest - result''' <BR> '''Beta - result'''  
| ashughes<BR> '''Betatest - result''' <BR> '''releasetest - result''' <BR> '''Beta - result'''  
| hskupin<BR> '''Betatest - result''' <BR> '''releasetest - result''' <BR> '''Beta - result'''  
| juanb/ashughes<BR> '''Betatest - result''' <BR> '''releasetest - result''' <BR> '''Beta - result'''
| juanb/ashughes<BR> '''Betatest - result''' <BR> '''releasetest - result''' <BR> '''Beta - result'''
|-
|-
Line 74: Line 86:
|-
|-
| Staging/Live Webpages & Spotchecks & Webpages
| Staging/Live Webpages & Spotchecks & Webpages
| tester - '''result'''
| juanb - '''result'''
| tester - '''result'''
| tester - '''result'''
| tester - '''result'''
| tester - '''result'''
Line 136: Line 148:
=== Willing to help? Sign Up Here ===
=== Willing to help? Sign Up Here ===
* nickname <email (optional)> <details>
* nickname <email (optional)> <details>
* stechz (Benjamin Stover): Mac OS 10.5.6


= <FONT COLOR = "RED">Detailed Results </FONT> =
= <FONT COLOR = "RED">Detailed Results </FONT> =


== Smoketests Results ==
== Smoketests Results - Build 1 ==
=== Windows XP [tester] ===
=== Windows XP [tracy] ===
=== Mac OS X [hskupin] [marcia] ===
* Status: '''PASS''' (6/05/09)
* Build: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b99) Gecko/20090605 Firefox/3.5b99
* ID:20090605010243
 
* Bugs seen:
** [https://bugzilla.mozilla.org/show_bug.cgi?id=490009 bug 490009] Migration doesn't fire (this seems to be local to a few machines, which we can't sort out why)
 
=== Mac OS X [hskupin] [marcia] [ashughes] ===
* Status: '''PASS''' (6/05/09)
* Status: '''PASS''' (6/05/09)
* Build: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b99) Gecko/20090604 Firefox/3.5b99 ID:20090604233133
* Build: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b99) Gecko/20090604 Firefox/3.5b99 ID:20090604233133
Line 151: Line 171:
* Bugs Filed:
* Bugs Filed:
** [https://bugzilla.mozilla.org/show_bug.cgi?id=496577 Bug 496577] "Sort by Name" menu item greyed out for RSS feeds
** [https://bugzilla.mozilla.org/show_bug.cgi?id=496577 Bug 496577] "Sort by Name" menu item greyed out for RSS feeds
** [https://bugzilla.mozilla.org/show_bug.cgi?id=496594 Bug 496594]  [nsIXMLHttpRequest.send] error while running Geolocation test in PB mode
** [https://bugzilla.mozilla.org/show_bug.cgi?id=496616 Bug 496616]  Security icon in status bar too difficult to open
* ashughes' testing - Mac OSX Tiger Intel
** Status: '''PASS''' (6/05/09)
** Build Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9.1b99) Gecko/20090604 Firefox/3.5b99
** Bugs seen:
*** [https://bugzilla.mozilla.org/show_bug.cgi?id=391981 Better minimum size for browser window]
=== Linux [juanb] ===
* Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1b99) Gecko/20090605 Firefox/3.5b99
* Build ID: 20090604233331
* Built from: http://hg.mozilla.org/releases/mozilla-1.9.1/rev/0d4036dfff71
** Issues: Video playback problems reported in TestDay channel
== Smoketests Results - Build 2 ==
=== Windows XP [tracy] ===
* Status: '''PASS''' (6/06/09)
* Build: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b99) Gecko/20090605 Firefox/3.5b99
* ID: 20090605162636
* Bugs seen:
** [https://bugzilla.mozilla.org/show_bug.cgi?id=490009 bug 490009] Migration doesn't fire (this seems to be local to a few machines, which we can't sort out why)
=== Mac OS X [marcia/juanb] ===
* Status: '''PASS''' (6/05/09)
* Build:  Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b99) Gecko/20090605 Firefox/3.5b99  ID: 20090605144508
* Issues noted:
**Difficult to tear away a tab to the desktop
**No animation shown when reloading an RSS feed
* Status: '''PASS''' (MacIntel 10.4.11) - juanb - (6/08)
* Build: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9.1b99) Gecko/20090605 Firefox/3.5b9
* Build ID: 20090605144508
* Issues:
* PPC Mac (Leopard) Spotcheck - '''PASS''' - marcia - (6/08)
* Build: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.5; en-US; rv:1.9.1b99) Gecko/20090605 Firefox/3.5b99
* Build ID: 20090605144508
* Issues:
**Crashing when print dialog open [HPPrintSettings@0x23c90] ->Bug filed https://bugzilla.mozilla.org/show_bug.cgi?id=496973


=== Linux [tester] ===
=== Linux [juanb] ===
* Status: '''PASS''' 6/6
* Build: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1b99) Gecko/20090605 Firefox/3.5b99
* Build ID: 20090605144815
* Built from: http://hg.mozilla.org/releases/mozilla-1.9.1/rev/3f775c3bb24a
** Issues:


== Testday Results ==
== Testday Results ==
* BetaPreview Testday scheduled June 5th
* BetaPreview Testday scheduled June 5th
* ashughes:
** I primarily focused on Session Store testing ([https://wiki.mozilla.org/QA/Firefox3.5/TestPlan/BetaPreview/MocoTestday/SessionRestore Test Plan])
** Blockers:
*** [https://bugzilla.mozilla.org/show_bug.cgi?id=497482 bug 497482] - FIXED 1.9.1
** Session Store Bugs:
*** [https://bugzilla.mozilla.org/show_bug.cgi?id=497658 bug 497658] - Regression (3.5b4 20090415)
*** [https://bugzilla.mozilla.org/show_bug.cgi?id=497730 bug 497730] - Loading Tabs become about:blank after crash
*** [https://bugzilla.mozilla.org/show_bug.cgi?id=497752 bug 497752] - Session Restore should remember search prefs
** Other Bugs:
*** [https://bugzilla.mozilla.org/show_bug.cgi?id=497715 bug 497715] - Latest Headlines overlaps Multiple Tab warning
*** [https://bugzilla.mozilla.org/show_bug.cgi?id=497723 bug 497723] - Alltabs dropdown lacks scroll arrows on Mac

Latest revision as of 01:26, 12 June 2009

« QA/Firefox3.5/TestPlan

Firefox 3.5 BetaPreview Test Results

This is a precursor build for RC1. We are calling it a Beta Preview because its got nearly the last bits of code, but not quite ready for RC1. Please test this build like its another beta, but know that its stable enough to be an RC.

Related Pages

Firefox 3.5 Tracking Site

Project contacts

PM: Mike Beltzner (beltzner)
Build: Nick Thomas (nthomas)
QA: Juanb

Schedule

Code Freeze
6/4
Builds hand to QA
6/5
QA test signoff
?
  • See Test Coverage below

Build Location and IDs (Build 2)

Download:

Builds:

Test Coverage

  • Areas Covered:
    • Smoketests
    • L10n (limited)
    • Updates (limited)
    • Spotchecks & Webpages
    • Bug Fixes
  • Areas Not Covered:
    • BFTs
    • FFTs (will carry over into nightly testing, but not dependent on rc1 ship date)
    • Topsites (covered by Tomcat's automation)
    • Top Financial Sites (covered by website project)
    • Top Extensions & Themes
    • Top Plugins
    • Accessibility
    • Updates
    • Spotchecks & Webpages
    • Bug Fixes

Test Results Build2

Types of Tests Windows XP Mac OSX Linux
Smoketests tracy - PASS with noted migration bug marci/juanb/hskupin - PASS pending result on ppc machine juanb - PASS
L10n tchung - PASS tchung - PASS tchung - PASS
Updates tracy
Betatest - result
releasetest - result
Beta - result 		hskupin
Betatest - result
releasetest - result
Beta - result 		juanb/ashughes
Betatest - result
releasetest - result
Beta - result
Additional Focused Testing QA team - result QA team - result QA team - result
Staging/Live Webpages & Spotchecks & Webpages juanb - result tester - result tester - result

Smoketest Testing

Lite verification tests (or smoketests) will be first ran when the builds are ready. Tests are available on litmus under the Firefox 3.5 test suite. In any case the Smoketests result, we will raise blockers asap before proceeding with the bfts.

Please record your detailed results in the section below.

Updates Testing

We will be doing updates of all Alpha/Beta users to BetaPreview. We will be checking complete updates, and failover scenarios.

Will be available on betatest, releasetest, and beta channels.

Testcases and Results are tracked here.

L10N

Visual inspection: Go through the P1 list of locales, and run a spotcheck to see if everything visually looks fine. Refer to Fx3.5 L10N Requirements for more specifics.

Testcases and Results are tracked here.

Staging/Live Spotchecks & Webpages

1) Verify the following webpages that gets pushed to staging and then to the live sites. Look for content messaging, spelling mistakes, broken links & imnages, etc..

  • Firstrun page ([Staging link])
  • Whatsnew Page ([Staging link])
  • Releasenotes page ([Staging link])

2) Check live site download links on all platforms. Please cover en-US, and 2 or 3 additional Tier 1 locales as well. Pay attention to the following areas:

  • Check for signed installer and applications (windows only)
  • Firstrun page, whatsnew page, Home page, Search plugin
  • Install, uninstall
  • General spotchecks and browser functionality
  • Confirm Build IDs

Additional Resource Testing

Given the minimized focus testing around BetaPreview due to time constraints, there are a few regression areas that will need some focus testing. This area will call for additional resources to help QA in this effort to ship BetaPreview in a timely manner.

Fixed Bugs Verifications

The following bugs are fixed for BetaPreview and need verifications:

Testday

A BetaPreview testday will be held and call for the community support to assist with testing. Activities include executing litmus testcases, bug verifications, and focus on the new feature areas that are being implemented. More can be found on QMO.

Willing to help? Sign Up Here

  • nickname <email (optional)> <details>
  • stechz (Benjamin Stover): Mac OS 10.5.6

Detailed Results

Smoketests Results - Build 1

Windows XP [tracy]

  • Status: PASS (6/05/09)
  • Build: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b99) Gecko/20090605 Firefox/3.5b99
  • ID:20090605010243
  • Bugs seen:
    • bug 490009 Migration doesn't fire (this seems to be local to a few machines, which we can't sort out why)

Mac OS X [hskupin] [marcia] [ashughes]

  • Status: PASS (6/05/09)
  • Build: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b99) Gecko/20090604 Firefox/3.5b99 ID:20090604233133
  • Bugs seen:
    • bug 387834 Minefield steals default browser preference/setting from Bon Echo/branch builds
  • marcia's testing
  • Build Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b99) Gecko/20090604 Firefox/3.5b99
  • Bugs Filed:
    • Bug 496577 "Sort by Name" menu item greyed out for RSS feeds
    • Bug 496594 [nsIXMLHttpRequest.send] error while running Geolocation test in PB mode
    • Bug 496616 Security icon in status bar too difficult to open
  • ashughes' testing - Mac OSX Tiger Intel

Linux [juanb]

Smoketests Results - Build 2

Windows XP [tracy]

  • Status: PASS (6/06/09)
  • Build: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b99) Gecko/20090605 Firefox/3.5b99
  • ID: 20090605162636
  • Bugs seen:
    • bug 490009 Migration doesn't fire (this seems to be local to a few machines, which we can't sort out why)

Mac OS X [marcia/juanb]

  • Status: PASS (6/05/09)
  • Build: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b99) Gecko/20090605 Firefox/3.5b99 ID: 20090605144508
  • Issues noted:
    • Difficult to tear away a tab to the desktop
    • No animation shown when reloading an RSS feed
  • Status: PASS (MacIntel 10.4.11) - juanb - (6/08)
  • Build: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9.1b99) Gecko/20090605 Firefox/3.5b9
  • Build ID: 20090605144508
  • Issues:
  • PPC Mac (Leopard) Spotcheck - PASS - marcia - (6/08)
  • Build: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.5; en-US; rv:1.9.1b99) Gecko/20090605 Firefox/3.5b99
  • Build ID: 20090605144508
  • Issues:

Linux [juanb]

Testday Results

  • BetaPreview Testday scheduled June 5th
  • ashughes:
    • I primarily focused on Session Store testing (Test Plan)
    • Blockers:
    • Session Store Bugs:
      • bug 497658 - Regression (3.5b4 20090415)
      • bug 497730 - Loading Tabs become about:blank after crash
      • bug 497752 - Session Restore should remember search prefs
    • Other Bugs:
      • bug 497715 - Latest Headlines overlaps Multiple Tab warning
      • bug 497723 - Alltabs dropdown lacks scroll arrows on Mac
Retrieved from "https://wiki.allizom.org/index.php?title=QA/Firefox3.5/TestPlan/BetaPreview&oldid=148794"