MozillaWiki

QA/Fennec/Milestones/2.0/ReleaseTest/Beta3

< QA‎ | Fennec‎ | Milestones‎ | 2.0
Revision as of 13:54, 16 December 2010 by Anamaria.moldovan (talk | contribs) (→‎Droid 2 [Anna])

Fennec 4.0 Beta 3 Test Results

The covered testing for this release will focus on smoketests, verifications and updates. For a list of new features coming in for Fennec 4.0, take a look at the Mobile Planning Page and for test coverage, look at our test plan.

Project contacts

PM: Stuart Parmenter (stuart)
Build: John O'duinn (oduinn)
QA: Aakash Desai (aakashd)

Schedule

Code Freeze
--
Builds hand to QA
--
QA test signoff
--

Build Location and IDs

  1. Go to the candidates folder in our ftp folders
  2. Go to the "4.0b3-candidates" folder
  3. Download the candidate:
    • For the n900, install the application catalog and update to the candidate build via your device App Manager.
    • For Android, download and install the actual build over any other instance

Test Coverage

Smoketests

Lite verification tests (or smoketests) will be first run when the builds are ready. Tests are available on litmus under the Fennec 2.0 Test Suite. In any case of failure in any Smoketest, blockers need to be raised ASAP before proceeding to the BFTs.

Please record your detailed results in the "Detailed Test Results" section below.

BFTs for 2.0 Features

BFTs are a more comprehensive set of litmus tests that cover functional testing of features that have landed on Fennec 2.0. Tests are available on litmus under the Fennec 2.0 Test Suite.

Please record your detailed results in the "Detailed Test Results" section below.

Here are the BFT's that need to be run:

  • Panning & Zooming (Screen Navigation)
  • Form Assistant
  • Sync

OS Integration

The Operating System Integration testsuite is a set of tests that target how Fennec works with the platform (i.e. Android, Maemo, etc.) and features of the device itself (i.e. Orientation, hard keyboard, soft keyboard, etc.). Tests are available on litmus under the Fennec 2.0 Test Suite under the "OS Integration Tests" testgroup.

Please record your detailed results in the "Detailed Test Results" section below.

Fixed Bugs Verifications

The following link contains a list of bugs that are fixed for beta 3 and need verifications:

L10n Checks

We don't have coverage of localized builds until the candidate builds are available, so we'll need to run through the tests within the Fennec 2.0 String Guide on the following locales:

  • German
  • Finnish
  • French

Updates

Test the update path for the released builds only, not the nightlies. (eg. 1.1->4.0b3 (maemo only), 4.0b1->4.0b3, and 4.0b2->4.0b3)

Fennec 2.0 Beta 3 can be downloaded from the Mozilla Beta Catalog.

Updates tests to perform for each build

Mobile War Room Testing

We will be conducting a Testing and Feedback session for android users during the beta 3 release process. The purpose here is to spend some time using the product and looking at high level tests.

List of Testcases and Results

Test Results

Types of Tests Maemo Android
Smoketests
  • ahoza - DONE
  • Ioana - DONE
BFTs
  • ahoza - DONE
  • ashah - result
OS Integration
  • tchung martijn - result
  • Ioana - DONE
L10n
  • ahoza - DONE
  • Ioana - result
Updates
  • ahoza - DONE
  • tchung martijn - result
  • Ioana - result
  • anna - DONE
  • tchung - Pass

Detailed Test Results

Smoketests Results

HTC Desire [ioana]

  • Build: Mozilla/5.0 (Android; Linux armv7l; rv:2.0b8pre) Gecko/20101214 Firefox/4.0b8pre Fennec/4.0b3pre
  • Results: DONE
  • Notes:
    • the update doesn't appear neither from alpha, nor from 4.0b1/4.0b2
    • bug 619308 Password can be seen after a failed login
    • ghost tabs appear when sharing a link through Facebook - not always

n900 [ahoza]

  • Build: Mozilla/5.0 (Maemo; Linux armv7l;rv:2.0b8pre) Gecko/20101214 Firefox/4.08pre Fennec/4.0b3
  • Results: PASS
  • Notes:
    • No new issues found

BFT Results

Nexus One [ashah]

  • Build: Build: Mozilla/5.0 (Android; Linux armv7l; rv:2.0b8pre) Gecko/20101214 Firefox/4.0b8pre Fennec/4.0b3
  • Results: result
  • Notes:

n900 [ahoza]

  • Build: Mozilla/5.0 (Maemo; Linux armv7l;rv:2.0b8pre) Gecko/20101214 Firefox/4.08pre Fennec/4.0b3
  • Results: PASS
  • Notes:
    • Form Assistant
    • Sync
    • Screen Navigation
      • No blocking issues found

OS Integration Results

HTC Desire [Ioana]

  • Build: Mozilla/5.0 (Android; Linux armv7l; rv:2.0b8pre) Gecko/20101214 Firefox/4.0b8pre Fennec/4.0b3pre
  • Results: DONE
  • Notes:
    • Known Bugs:
      • Blocking Beta3+bug 619308 Password can be seen after a failed login
      • bug 617555 System menu button closes the VKB
      • bug 615166 Search button in landscape mode has no caption
      • bug 608340 After dismissing the VKB once, you cannot bring up the VKB again by double clicking the awesome bar
      • bug 609600 Form helper restrict tab panel to one column
    • Unreproducible Issues:
      • Application freezes in Preferences after entering the sync section
      • Using Menu system button on HTC has different result than on other device: sometimes the url bar remains for this test case. Also pressing long menu system button will pop up the VKB and the URL remains always in front (test case fails )

n900 [martijn]

  • Build:
  • Results: result
  • Notes:

L10n Results

Android [ioana]

  • Build:
  • Results: result
  • Notes:

n900 [ahoza]

  • Build: Mozilla/5.0 (Maemo; Linux armv7l;rv:2.0b8pre) Gecko/20101214 Firefox/4.08pre Fennec/4.0b3
  • Results: result

Updates Results

HTC Desire [ioana]

  • Build: Mozilla/5.0 (Android; Linux armv7l; rv:2.0b8pre) Gecko/20101214 Firefox/4.0b8pre Fennec/4.0b3pre
  • Results: FAIL
  • Notes:
    • No update is present when configuring the application neither or Alpha, 4.0b1 nor 4.0b2
    • The Check for updates buttons doesn't trigger any action

Droid 2 [Anna]

  • Build: Mozilla/5.0 (Android; Linux armv7l; rv:2.0b8pre)Gecko/20101214 Firefox/4.0b8pre Fennec/4.0b3
  • Results: PASS
  • Notes:
    • Test Case 13530 [Android Only][Update]Android Update with Auto Confirmation Setup is unclear/broken. I could not obtain the update, even setting the property app.partner.test = test in about:config.

Nexus One [Tchung]

  • 4.0b2 -> 4.0b3 en-US
    • Results: PASS
    • Notes: bug 619444 was filed against the beta-cck-test channel, and was fixed and verified. Need to a new String in about:config; app.partner.test = test in order to make it work.
  • 4.0b1 -> 4.0b3 en-US
    • Results: PASS
    • Notes: bug 619444 was filed against the beta-cck-test channel, and was fixed and verified. Need to a new String in about:config; app.partner.test = test in order to make it work.

n900 [ahoza]

  • Build: Mozilla/5.0 (Maemo; Linux armv7l;rv:2.0b8pre) Gecko/20101214 Firefox/4.08pre Fennec/4.0b3
  • Results: PASS
  • Notes:

n900 [martijn]

  • Build:
  • Results: result
  • Notes:
Retrieved from "https://wiki.allizom.org/index.php?title=QA/Fennec/Milestones/2.0/ReleaseTest/Beta3&oldid=273952"