QA/Firefox3.5/TestPlan/Beta1

« QA/Firefox3.1/TestPlan

Firefox 3.1 Beta1 Test Results

The covered testing for this release will focus on smoketests, bfts, l10n and spotchecks. This will only be on the en-US locale.

Related Pages

Firefox 3.1 Tracking Site

• Beta 1 features

Contact Info

PM: Mike Beltzner (beltzner)
Build: John O'Duinn (joduinn)
QA: Tony / others

Schedule

Code Freeze

9/30

Builds hand to QA

10/8

QA test signoff

10/14

• Smoketests
• BFTs
• L10n
• Updates
• Spotchecks

Build Location and IDs

Download:

• http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/3.1b1-candidates/build1
• ftp://ftp.mozilla.org/pub/firefox/nightly/3.1b1-candidates/build1/

Builds:

• Windows
  • Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1
    • 20081007144708
• Mac
  • Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1 (Full Build ID is: 20081007125523)
  • Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1
  • Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1
• Linux: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1

Test Coverage

• Areas Covered:
  • Smoketests
  • BFTs
  • L10n
  • Updates
  • Spotchecks & Webpages
  • Bug Fixes

• Areas Not Covered:
  • FFTs
  • Topsites
  • Accessibility

Test Results

Types of Tests	Windows XP	Mac 10.5	Linux
Smoketests	juanb - PASS	marcia - PASS	tomcat - PASS
BFTs	tracy - result	marcia - result	Joel - result
L10n	Tony - result	Tony - result	Tony - PASS
Updates	tracy Betatest - result Beta - result	marcia Betatest - result Beta - result	Tony Betatest - result Beta - result
Live Spotchecks & Webpages	juanb - result	marcia - result	Tony - result

Smoketest Testing

Lite verification tests (or smoketests) will be first ran when the builds are ready. Tests are available on litmus under the Firefox 3.1 test suite. In any case the Smoketests fail, we will raise blockers asap before proceeding with the bfts.

Please record your detailed results in the section below.

BFT Testing

BFTs are a more comprehensive set of litmus tests that cover the build testing after smoketests pass. Tests are available on litmus under the Firefox 3.1 test suite.

Please record your detailed results in the section below.

Updates Testing

We will be doing updates of all Alpha users to Beta1. We will be checking partial updates, complete updates, and failover scenarios.

Will be available on betatest, and beta channels.

Testcases and Results are tracked here.

L10N

Visual inspection: Go through the P1 list of locales, and run a spotcheck to see if everything visually looks fine. Refer to L10N Requirements for more specifics.

Testcases and Results are tracked here

Live Spotchecks & Webpages

1) Verify the following webpages that gets pushed to staging and then to the live sites. Look for content messaging, spelling mistakes, broken links & imnages, etc..

• Firstrun page
• Whatsnew Page
• Releasenotes page

2) Check live site download links on all platforms. Please cover en-US, and 2 or 3 additional Tier 1 locales as well. Pay attention to the following areas:

• Check for signed installer and applications (windows only)
• Firstrun page, whatsnew page, Home page, Search plugin
• Install, uninstall
• General spotchecks and browser functionality
• Confirm Build IDs

Fixed Bugs Verifications

The following bugs are fixed for Beta 1 and need verifications:

• Platform 1.9.1 bugs: http://tinyurl.com/4c4qwb
• Firefox 3.1 bugs: http://tinyurl.com/3tw7zw

Testday

A Beta1 testday will be held and call for the community support to assist with testing. Activities include executing litmus testcases, bug verifications, and focus on the new feature areas that are being implemented. More can be found on QMO.

Detailed Results

Smoketests Results

Windows XP [juanb]

• Windows XP SP3 (vm)
• Smoketests Results in Litmus - PASS
• Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1
  • 20081007144708
  • bug 459082 - (filed) migration wizard not coming up
  • bug 458579 - (existing) Feed tab missing in page info

Mac [marcia]

• 10.5 Results, PASS, results in Litmus
  • Build ID: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1
  • Bugs Filed
    • Bug 458968 - Firefox Mac Beta 1 candidate build is branded "Minefield"

Linux [tomcat on Fedora F9]

• PASS - Results in Litmus
  • Build ID: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1

BFT Results

Mac [marcia]

• Build ID: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1
• Bugs Filed:
  • Bug 459157 Exception in (NS_ERROR_INVALID_POINTER) [nsIPermissionManager.add] failure blocking images from Page Info
  • Bug 459256 Error: [Exception... "'Error: %82' when calling method: [nsIDOMEventListener::handleEvent]