QA/Fennec/Milestones/2.0/ReleaseTest/RC1

Fennec 4.0 RC 1 Test Results

The covered testing for this release will focus on smoketests, verifications and updates. For a list of new features coming in for Fennec 4.0, take a look at the Mobile Planning Page and for test coverage, look at our test plan.

Project contacts

PM: Stuart Parmenter (stuart)
Build: John O'duinn (oduinn)
QA: Aakash Desai (aakashd)

Schedule

Code Freeze

-- 3/16

Builds hand to QA

--

QA test signoff

--

Build Location and IDs

1. Go to the candidates folder in our ftp folders
2. Go to the "4.0rc-candidates" folder
3. Download the candidate:
   • For the n900, install the application catalog and update to the candidate build via your device App Manager.
   • For Android, download and install the actual build over any other instance

Test Coverage

Smoketests

Lite verification tests (or smoketests) will be first run when the builds are ready. Tests are available on litmus under the Fennec 2.0 Test Suite. In any case of failure in any Smoketest, blockers need to be raised ASAP before proceeding to the BFTs.

Please record your detailed results in the "Detailed Test Results" section below.

BFTs for 2.0 Features

BFTs are a more comprehensive set of litmus tests that cover functional testing of features that have landed on Fennec 2.0. Tests are available on litmus under the Fennec 2.0 Test Suite.

Please record your detailed results in the "Detailed Test Results" section below.

OS Integration

The Operating System Integration testsuite is a set of tests that target how Fennec works with the platform (i.e. Android, Maemo, etc.) and features of the device itself (i.e. Orientation, hard keyboard, soft keyboard, etc.). Tests are available on litmus under the Fennec 2.0 Test Suite under the "OS Integration Tests" testgroup.

Please record your detailed results in the "Detailed Test Results" section below.

Fixed Bugs Verifications

The following link contains a list of bugs that are fixed for RC and need verifications:

• 2.0+ Fennec blockers at a resolved state, but un-verified

We are holding a Fennec RC Bug Bash Session in Zombocom on Thursday, 3/17, at 1:30PM-3:00PM. In the meeting, all bugs in the query above will be verified until we get to a 0 state or, at least, all bugs that can be verified.

L10n Checks

We don't have coverage of localized builds until the candidate builds are available, so we'll need to run through the tests within the Fennec 2.0 String Guide on the following locales:

• German
• Finnish
• French

Top Add-ons Testing

To verify the entire browsing experience, QA will need to spotcheck for the following use cases across each add-on in the recommended add-ons list:

1. Installation through the Add-ons Manager or AMO
2. Spotcheck to make sure the UI is not entirely broken
3. Uninstall the add-on through the Add-ons Manager

Website & Device Stability Checks

The covered testing for this release candidate will include that of a test plan for website compatibility. Please read and use the website compatibility test plan, available here as a companion to a form that we would like you to complete, which is available here. Results are viewable here

Updates

Test the update path for the released builds only, not the nightlies. (eg. 1.1->4.0RC (maemo only) and 4.0b5->4.0)

Fennec 4.0 RC can be downloaded from the Mozilla Beta Catalog or the Android Marketplace.

Updates tests to perform for each build

Test Results

Types of Tests	Maemo	Android
Smoketests	Ioana - PASS	Anna - PASS
BFTs	Ioana - result	Anna - In progress
OS Integration	mw22 - PASS	nhirata - result
L10n	ashah - result	nhirata - result
Top Add-ons	aakashd - result	ashah - result
Fixed Bug Verifications	All	All
Website and Device Stability Checks	--	AaronMT - result
Updates	ashah - result	--

Detailed Test Results

Smoketests Results

HTC Desire [Anna]

• Build: Mozilla /5.0 (Android;Linux armv7l;rv:2.0b13pre) Gecko/20110317 Firefox/4.0b13pre Fennec /4.0
• Results: PASS

• Notes:
  • No unknown issues were found
  • Known bugs:
    • bug 628269 : Pages are zoomed in at reload
    • bug 639504 : Unable to zoom in/out a webpage on header/footer when a double tap gesture is made

n900 [Ioana]

• Both RC 1 and RC 2
• Build: Mozilla /5.0 (Maemo; Linux armv7l;rv:2.1) Gecko/20110317 Firefox/4.0b13pre Fennec /4.0
• Results: PASS

• Notes:
  • No major issue found
  • Known bugs:
    • bug 611294 - After sharing a page using Facebook, the browser returns to the shared page.
    • bug 639504 : Unable to zoom in/out a webpage on header/footer when a double tap gesture is made

BFT Results

HTC Desire [Anna]

• Build: Mozilla /5.0 (Android;Linux armv7l;rv:2.1) Gecko/20110317 Firefox/4.0b13pre Fennec /4.0
• Results: In progress

• Notes:
  • Options (Preferences):
    • There is no feedback pannel in RC1 Build 2. Could this be related with bug 642718 ?
  • General
    • When google mobile version webpage is loaded there arem't any suggestions offered. (Google suggestions are offered only in the classic version)
    • known bugs:
      • bug 628269: Pages are zoomed in at reload
      • bug 629645: Strange behavior when using geolocation pages.
      • bug 639829: Page position is not persistent on reload
  • Downloading
    • known bugs:
      • bug 619054: Inconsistency at auto-renaming
  • Tabbed Browsing
    • known bugs:
      • bug 639794: Opening a webpage from an external app should hide sidebars and panels
  • Find in Page
    • reported bugs:
      • bug 642788: [Regression] Find in Page does not scroll down when hitting the Down Arrow from the findbar

n900 [ashah]

• Build:
• Results: result

• Notes:

OS Integration Results

Nexus One [AaronMT]

• Build:
  • Mozilla/5.0 (Android; Linux armv7l; rv:2.1) Gecko/20110317 Firefox/4.0b13pre Fennec/4.0
• Results: PASS

• Notes:

n900 [aakashd] [RC1 build 2 - mw22]

• Build: Mozilla/5.0 (Maemo; Linux armv7l; rv:2.1) Gecko/20110317 Firefox/4.0b13pre Fennec/4.0
• Results: PASS

• Notes:
  • Still see bug 624691 - [Maemo-only] 'Open' button is disabled for downloaded files with .deb, .install and .pdf extension

RC1 build 2:

• Build: Mozilla/5.0 (Maemo; Linux armv7l; rv:2.1) Gecko/20110317 Firefox/4.0b13pre Fennec/4.0

• Results: PASS

• Notes:
  • Major issue: bug 642571 - UI bustage after a while on Maemo (no steps to reproduce yet). This happens very occassionally, I've seen it 3 times now.
  • bug 642750 - [Maemo] Tapping second time on url bar in landscape mode does not bring up virtual keyboard
  • bug 642771 - [Maemo] Dismissing the virtual keyboard causes the top bar with "All pages", "Bookmarks" and "History" to disappear, part 2
  • I'm also seeing bug 624691 - [Maemo-only] 'Open' button is disabled for downloaded files with .deb, .install and .pdf extension

L10n Results

Motorola Droid 2 [nhirata]

• Build:
• Results: result

• Notes:

n900 [Ioana]

• Build:
• Results: result

• Notes:

Top Add-ons Results

n900 [ashah]

• Build:
• Results: result

• Notes:
  • Found these bugs, while testing RC1: (from mw22)
  • bug 642642 Exception... "'Find cluster denied: illegal-method' when calling method: [nsIObserver::observe] in resource://services-sync/ext/Observers.js:122
  • bug 642635 Adding/removing a bookmark isn't reflected in the awesomebar results popup
  • bug 642571 UI bustage after a while on Maemo (no steps to reproduce yet)

n900 [aakashd]

• Build: Mozilla/5.0 (Maemo; Linux armv7l; rv:2.1) Gecko/20110317 Firefox/4.0b13pre Fennec/4.0
• Results: PASS

• Notes:
  • Installed and Uninstalled the following addons: Phony, Readability, Personas, Bigger Text, Easy Reading, Quit Firefox for Mobile, Reading List, URL Fixer

Updates Results

Nexus One [AaronMT]

• Build:
• Results: result

• Notes:

n900 [Ioana]

• Build:
• Results: result

• Notes: