QA/Firefox3/TestPlan/Vista Integration

« QA/Firefox3/TestPlan

Firefox 3 Windows Vista Test Plan

Maintained by: Marcia Knous and Tomcat

• First Draft, 09/07/07
• Revised 10/23/07
• Second Revision 2/15/08

Overview

The primary goal of this QA test plan is the testing of the OS integration piece of Firefox 3 as it relates to Windows Vista. This is make sure that existing functionality works and that all newly supported functionality works (primarily support of parental controls)

Scope of Planned Testing

Testing will cover the areas below. Exceptions are noted under the Will not test section.

• Installing and Uninstalling Firefox on Vista
  • XPInstall Vista Permission problems
  • PRD item: Support a .exe installer on Windows
  • PRD item: Support for "scripted" installations (ie, command line options)
  • PRD item: Support for silent install (no UI)
  • PRD item: Support replacing images in the wizard
  • Installing in the Default Directory
  • Installing in a non-default directory
  • Uninstalling Firefox
  • Confirming that the "Tell Us about Firefox" survey is invoked after uninstalling
  • Installing Firefox as a Admin User
  • Installing Firefox as a non-Admin User
  • Confirming build has been signed
• Setting Firefox as the Default Application
  • Resetting IE as the default, then setting Firefox as the default
  • Setting FF as the default from the install
  • Setting FF as the default from Desktop Icon or Start | Programs
  • Setting FF as the default after a software update
  • Setting FF as the default before a software update
• Confirming that we shut down cleanly and no active .exe processes are running
  • Running as an Admin User
  • Running as a Standard User
  • Running with shadow copies enabled
• Running on Vista as an Admin User
  • With Parental Controls Invoked
• Running on Vista as a Standard User
  • With Parental Controls Invoked
• Software Update
  • Updating as a Admin user
  • Updating as a Standard User
  • Confiming Major/Minor Update functionality
  • Automatic Update with limited account privileges
  • Confirming that updater.exe has been signed
• RSS Feed Reader Client Side Applications
• Downloading files to the desktop
  • Honoring whatever is set as the default download location on Vista (https://bugzilla.mozilla.org/show_bug.cgi?id=355554)
• Fonts
• Plugins
  • Flash
  • Shockwave
  • Quicktime
  • Windows Media Player
  • PDFs
  • Java
  • RealPlayer
  • Viewpoint
• Add-ons
  • Installing
  • Uninstalling
  • Enabling and Disabling
  • Finding Updates
  • Add-ons Compatibility checks
• Visual Refresh (Vista icons)
  • Runtime check to make sure we serve up Vista icons Bug 416531
• Service Pack 1
  • RTM in March?

Will Not Test

Will NOT test

As I understand if from talking with Rob Strong, Windows Vista and XP should exhibit the same types of bugs in the browser space. The area of focus for Vista need to be on the pieces that are touch or work in concert with the OS. Therefore it will not be necessary to do exhaustive testing of browser functionality. We will keep the same strategy we do for all of our major releases and do a thorough full functional test of the release.

Platforms and Configurations

At the time of this writing, Windows Vista ships several different versionS of Vista, including:

• Vista Ultimate
• Vista Home Basic
• Vista Home Premium
• Vista Business

Parental Controls is not shipped with the lower end versions of Vista, so emphasis should be given to Vista Ultimate and Home Premium to make sure PC is fully covered.

It is not likely that QA will be able to cover all of these editions.

Major Test Areas

Our Test coverage will be a multi-leveled approach:

• 1. Functional testing and
• 2. Some stress/boundary/negative tests (depth of functionality):
  • FFT in Litmus run on the Windows Vista platform
• 3. Ad hoc and regression testing

Functional Testing

• We will run the Litmus Win Vista specific testing suite.
• Litmus test cases will be added for any new Vista specific features that are supported.

Stress testing/Negative Testing(depth of functionality):

Due to the fact the Vista is considerably different than XP, we should do a fair amount of negative testing to expose as many Vista-specific issues as possible.

Ad hoc & regression testing

The Mozilla community users will cover some of the Ad hoc testing. Regression testing will cover:

• QA Verification of Resolved bugs
• Daily smoketesting of builds

References to Relevant Bugs (not complete)

• Parental Controls can be easily bypassed by going back to the download source
• Vista Tracking Bug
• Dependency Tree
• File downloads should honor Vista's parent control setting
• Honor URL override settings for Vista's parental control settings
• Change default downloading folder in Windows Vista from Desktop to Downloads
• ui problems when attempting to run a .exe from the download manager on vista
• disabled html selects have blue background on vista
• Widget Animation on Vista?
• On Vista, every menulist should be a menubutton
• avoid the second UAC prompt for helper.exe on software update by launching it directly from the elevated updater.exe process
• Standard Vista account with Parental Controls turned on cannot launch Firefox
• Handle OS Shutdown sanely on Windows/Linux (dataloss/odd UI)
• Ability to install as a standard user on Vista is not available
• Set the default app on Vista via the installer

Fixed Bugs that need to be verified

• 399381 Support Custom Wizard Images
• 398434 - Provide option to remove profiles during uninstall -> VERIFIED
• 393149 - Some registry keys aren't removed under HKEY_LOCAL_MACHINE\SOFTWARE\Mozilla\ -> VERIFIED
• 370571 - Ability to install as a standard user on Vista is not available -

Bugs to Keep an eye on

• [https://bugzilla.mozilla.org/show_bug.cgi?id=396509

Standard Vista account with Parental Controls turned on cannot launch Firefox]

Litmus test cases added or changed

• Verify Firefox launches when running as Standard User with Parental Controls on

Schedule/milestones

( Provide some breakdown of the major areas of the feature, how long it will take to develop or run the tests, and an approximate time this will be done.