QA/Fennec/Milestones/2.0/ReleaseTest/Beta3

Fennec 4.0 Beta 3 Test Results

The covered testing for this release will focus on smoketests, verifications and updates. For a list of new features coming in for Fennec 4.0, take a look at the Mobile Planning Page and for test coverage, look at our test plan.

Project contacts

PM: Stuart Parmenter (stuart)
Build: John O'duinn (oduinn)
QA: Aakash Desai (aakashd)

Schedule

Code Freeze

--

Builds hand to QA

--

QA test signoff

--

Build Location and IDs

1. Go to the candidates folder in our ftp folders
2. Go to the "4.0b3-candidates" folder
3. Download the candidate:
   • For the n900, install the application catalog and update to the candidate build via your device App Manager.
   • For Android, download and install the actual build over any other instance

Test Coverage

Smoketests

Lite verification tests (or smoketests) will be first run when the builds are ready. Tests are available on litmus under the Fennec 2.0 Test Suite. In any case of failure in any Smoketest, blockers need to be raised ASAP before proceeding to the BFTs.

Please record your detailed results in the "Detailed Test Results" section below.

BFTs for 2.0 Features

BFTs are a more comprehensive set of litmus tests that cover functional testing of features that have landed on Fennec 2.0. Tests are available on litmus under the Fennec 2.0 Test Suite.

Please record your detailed results in the "Detailed Test Results" section below.

Here are the BFT's that need to be run:

• Panning & Zooming (Screen Navigation)
• Form Assistant
• Sync

OS Integration

The Operating System Integration testsuite is a set of tests that target how Fennec works with the platform (i.e. Android, Maemo, etc.) and features of the device itself (i.e. Orientation, hard keyboard, soft keyboard, etc.). Tests are available on litmus under the Fennec 2.0 Test Suite under the "OS Integration Tests" testgroup.

Please record your detailed results in the "Detailed Test Results" section below.

Fixed Bugs Verifications

The following link contains a list of bugs that are fixed for beta 3 and need verifications:

• 2.0b3+ Fennec blockers at a resolved state, but un-verified

L10n Checks

We don't have coverage of localized builds until the candidate builds are available, so we'll need to run through the tests within the Fennec 2.0 String Guide on the following locales:

• German
• Finnish
• French

Updates

Fennec 2.0 Beta 3 can be downloaded from the Mozilla Beta Catalog.

Updates tests to perform for each build

Mobile War Room Testing

We will be conducting a Testing and Feedback session for android users during the beta 3 release process. The purpose here is to spend some time using the product and looking at high level tests.

List of Testcases and Results

Test Results

Types of Tests	Maemo	Android
Smoketests	ahoza - DONE	Ioana - DONE
BFTs	ahoza - DONE	ashah - result
OS Integration	tchung martijn - result	Ioana - DONE
L10n	ahoza - result	Ioana - result
Updates	ahoza - result tchung martijn - result	Ioana - result anna - result

Detailed Test Results

Smoketests Results

HTC Desire [ioana]

• Build: Mozilla/5.0 (Android; Linux armv7l; rv:2.0b8pre) Gecko/20101214 Firefox/4.0b8pre Fennec/4.0b3pre
• Results: DONE

• Notes:
  • the update doesn't appear neither from alpha, nor from 4.0b1/4.0b2
  • bug 619308 Password can be seen after a failed login
  • ghost tabs appear when sharing a link through Facebook - not always

n900 [ahoza]

• Build: Mozilla/5.0 (Maemo; Linux armv7l;rv:2.0b8pre) Gecko/20101214 Firefox/4.08pre Fennec/4.0b3
• Results: PASS

• Notes:
  • No new issues found

BFT Results

Nexus One [ashah]

• Build:
• Results: result

• Notes:

n900 [ahoza]

• Build: Mozilla/5.0 (Maemo; Linux armv7l;rv:2.0b8pre) Gecko/20101214 Firefox/4.08pre Fennec/4.0b3
• Results: PASS

• Notes:
  • Form Assistant
    • No blocking issues found
    • New bugs: bug 619299
  • Sync
    • No blocking issues found
    • Known bugs: bug 618858
  • Screen Navigation
    • No blocking issues found

OS Integration Results

HTC Desire [Ioana]

• Build: Mozilla/5.0 (Android; Linux armv7l; rv:2.0b8pre) Gecko/20101214 Firefox/4.0b8pre Fennec/4.0b3pre
• Results: DONE

• Notes:
  • Known Bugs:
    • bug 619308 Password can be seen after a failed login
    • bug 617555 System menu button closes the VKB
    • bug 615166 Search button in landscape mode has no caption
    • bug 608340 After dismissing the VKB once, you cannot bring up the VKB again by double clicking the awesome bar
    • bug 609600 Form helper restrict tab panel to one column
  • Unreproducible Issues:
    • Application freezes in Preferences after entering the sync section
    • Using Menu system button on HTC has different result than on other device: sometimes the url bar remains for this test case. Also pressing long menu system button will pop up the VKB and the URL remains always in front (test case fails )

n900 [aakashd]

• Build:
• Results: result

• Notes:

L10n Results

Droid [AaronMT]

• Build:
• Results: result

• Notes:

n900 [nhirata]

• Build:
• Results: result

• Notes:

Updates Results

HTC Desire [ioana]

• Build:
• Results: result

• Notes:

Droid 2 [Anna]

• Build: Mozilla/5.0 (Android; Linux armv7l; rv:2.0b8pre)Gecko/20101214 Firefox/4.0b8pre Fennec/4.0b3
• Results: FAIL

• Notes:
  • Last alpha release, 4.0b1 and 4.0b2 candidates and releases are NOT able to update to 4.0b3 build 1 using Litmus test cases for beta.

n900 [ahoza]

• Build:
• Results: result

• Notes:

n900 [aakashd]

• Build:
• Results: result

• Notes: