QA/Firefox3.5/TestPlan/Beta3

From MozillaWiki
Jump to navigation Jump to search

« QA/Firefox3.1/TestPlan

Firefox 3.1 beta3 Test Results

The covered testing for this release will focus on smoketests, bfts, l10n, top plugins, top financial sites, updates, and spotchecks.

Related Pages

Firefox 3.1 Tracking Site

Contact Info

PM: Mike Beltzner (beltzner)
Build: John O'Duinn (joduinn)
QA: tester / others

Schedule

Code Freeze
3/3
Builds hand to QA
3/5
QA test signoff
3/11
  • See Test Coverage below

Build Location and IDs

Download:

Builds:

  • Build 1:
    • Mac 10.5: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b3) Gecko/20090304 Firefox/3.1b3
    • Mac 10.4: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9.1b3) Gecko/20090304 Firefox/3.1b3
    • Windows: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b3) Gecko/20090304 Firefox/3.1b3
    • Linux: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1b3) Gecko/20090304 Firefox/3.1b3 (20090304215011)
  • Build 2:
    • Mac 10.5: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b3) Gecko/20090305 Firefox/3.1b3
    • Mac 10.4: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9.1b3) Gecko/20090305 Firefox/3.1b3

Test Coverage

  • Areas Covered:
    • Smoketests
    • BFTs
    • L10n
    • Top Plugins
    • Top Financial Sites
    • Updates
    • Spotchecks & Webpages
    • Bug Fixes
  • Areas Not Covered:
    • FFTs
    • Topsites (covered in nightly automation)
    • Top Themes
    • Accessibility

Test Results

Types of Tests Windows XP Mac OSX Linux
Smoketests Henrik - result Aakash - PASS juanb - PASS
BFTs Aakash/Henrik - result marcia - result <open> - result
L10n Henrik - result Tony - result Tony - result
Top Plugins Aakash - result Murali - result Murali - result
Top Financial sites Tony - result Marcia - result Tony - result
Updates Henrik
Betatest - result
releasetest - result
Beta - result 		Murali
Betatest - result
releasetest - result
Beta - result 		Aakash/Tony
Betatest - result
releasetest - result
Beta - result
Staging/Live Webpages & Spotchecks & Webpages <web tester>, Henrik - result <web tester>, Aakash - result <web tester>, Tony - result

Smoketest Testing

Lite verification tests (or smoketests) will be first ran when the builds are ready. Tests are available on litmus under the Firefox 3.1 test suite. In any case the Smoketests result, we will raise blockers asap before proceeding with the bfts.

Please record your detailed results in the section below.

Note - For windows testing, please also make a smoketest pass against windows 7 beta

BFT / FFT Testing

BFTs are a more comprehensive set of litmus tests that cover the build testing after smoketests result. Tests are available on litmus under the Firefox 3.1 test suite.

FFTs are a full comprehensive set of litmus tests that extend coverage in areas that BFTs dont often get to. These are run at least once during the major milestone timeline.

If you are running a BFT or FFT, Please record your detailed results in the section below.

Updates Testing

We will be doing updates of all Alpha users to beta3. We will be checking complete updates, and resultover scenarios.

Will be available on betatest, releasetest, and beta channels.

Testcases and Results are tracked here.

L10N

Visual inspection: Go through the P1 list of locales, and run a spotcheck to see if everything visually looks fine. Refer to L10N Requirements for more specifics.

Testcases and Results are tracked here.


Top Plugins Testing

Select a few top plugins from AMO and perform the following testcases. (examples: Flash, Acrobat, Java, Real Player)

Testcases:

  • Enabling extensions and confirming web compatibility
  • Disabling or Uninstalling plugins and confirming proper messaging and behavior
  • Switch profiles and verify plugins are still working properly

Testcases and Results are tracked here.

Top Financial Sites Testing

  • Select a few top financial sites from here
  • Internal link is here.

Testcases:

  • Login/logout, verify functionality
  • Look for any layout discrepencies
  • Make a financial transaction if possible
  • Be sure to test in private browing mode also

Staging/Live Spotchecks & Webpages

1) Verify the following webpages that gets pushed to staging and then to the live sites. Look for content messaging, spelling mistakes, broken links & imnages, etc..

2) Check live site download links on all platforms. Please cover en-US, and 2 or 3 additional Tier 1 locales as well. Pay attention to the following areas:

  • Check for signed installer and applications (windows only)
  • Firstrun page, whatsnew page, Home page, Search plugin
  • Install, uninstall
  • General spotchecks and browser functionality
  • Confirm Build IDs

Fixed Bugs Verifications

The following bugs are fixed for beta 3 and need verifications:

Testday

A beta3 testday will be held and call for the community support to assist with testing. Activities include executing litmus testcases, bug verifications, and focus on the new feature areas that are being implemented. More can be found on QMO.

Detailed Results

Smoketests Results

Windows XP [Henrik]

  • Build: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b3) Gecko/20090304 Firefox/3.1b3
    • 20090304233338
  • Bugs:
    • [Henrik]: filed bug 481714 - "Always ask" for PDF files is not obeyed
    • [Henrik]: Tearing-off a tab to desktop shows denied cursor (bug 463088)

Mac [Aakash]

  • Build: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b3) Gecko/20090304 Firefox/3.1b3 ID:20090304214853
  • [tchung] Filed bug bug 481680 - [Mac OSX] Beta 3 build 1 are showing "Mozilla Firefox3.1b3" in window title
    • Blocker filed, fix will land in build 2 respin.
  • [aakash] Filed bug bug 481694 - Home Page not imported from Safari

Linux [Juan]

  • Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1b3) Gecko/20090304 Firefox/3.1b3
    • 20090304215011
  • Bugs:
    • Reopened (eventually re-Resolved) - bug 475066 - unable to detach tabs,
    • Opened new bug and nominated blocking 3.1 - bug 481693 - unable to detach tabs in Linux
    • Cannot (should not?) create web shortcuts. There is an existing bug, but I need to dig it out. It's also related to tabbed browsing (tear off).

Spotchecks [marcia]

  • Filed bug 481687 - Regression that affects only Mac

BFT Results

Windows XP [Henrik/Aakash]

Mac [Marcia]

  • Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b3) Gecko/20090304 Firefox/3.1b3
  • Flash version is Shockwave Flash 10.0 r12

Linux [juanb]

  • Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1b3) Gecko/20090305 Firefox/3.1b3
    • 20090305134136
    • Status: In progress
Retrieved from "https://wiki.allizom.org/index.php?title=QA/Firefox3.5/TestPlan/Beta3&oldid=132090"