QA/Fennec/Milestones/2.0/ReleaseTest/4.0.1

Fennec 4.0.1 Test Results

The covered testing for this release will focus on smoketests, verifications and updates. We are focusing on the targeted fixes for Firefox Mobile 4.0.1 such as l10n @font-face, devices with Android 2.3.3, check l10n fixes, etc.

Project contacts

PM: Stuart Parmenter (stuart)
Build: John O'duinn (oduinn)
QA: Kevin Brosnan (kbrosnan)

Schedule

Code Freeze

-- 4/14

Builds hand to QA

-- 4/15 or 4/18

QA test signoff

-- Aprox. 4/20

Build Location and IDs

1. Go to the candidates folder in our ftp folders
2. Go to the "4.0.1-candidates" folder
3. Download the candidate:
   • For the n900, install the application catalog and update to the candidate build via your device App Manager.
   • For Android, download and install the actual build over any other instance

Test Coverage

Smoketests

Lite verification tests (or smoketests) will be first run when the builds are ready. Tests are available on litmus under the Fennec 2.0 Test Suite. In any case of failure in any Smoketest, blockers need to be raised ASAP before proceeding to the BFTs.

Please record your detailed results in the "Detailed Test Results" section below.

BFTs for 2.0 Features

BFTs are a more comprehensive set of litmus tests that cover functional testing of features that have landed on Fennec 2.0. Tests are available on litmus under the Fennec 2.0 Test Suite.

Please record your detailed results in the "Detailed Test Results" section below.

OS Integration

The Operating System Integration testsuite is a set of tests that target how Fennec works with the platform (i.e. Android, Maemo, etc.) and features of the device itself (i.e. Orientation, hard keyboard, soft keyboard, etc.). Tests are available on litmus under the Fennec 2.0 Test Suite under the "OS Integration Tests" testgroup.

Please record your detailed results in the "Detailed Test Results" section below.

Fixed Bugs Verifications

The following link contains a list of bugs that are fixed for 4.0.1 and need verifications:

• Fixed Firefox Mobile 4.0.1 blockers even verified bugs should be checked to make sure the fix landed in the Mobile-2.0 tree.

• bug 626051
• bug 643038
• bug 643307
• bug 643836
• bug 644012
• bug 644613
• bug 644735
• bug 649115

L10n Checks

Check l10n switching and that basic functionality does not break.

• German
• Finnish
• French
• Japanese

Updates

Test the update path for the released builds only, not the nightlies. (eg. 1.1->4.0RC (maemo only) and 4.0b5->4.0)

Fennec 4.0 RC can be downloaded from the Mozilla Beta Catalog or the Android Marketplace.

Updates tests to perform for each build

Test Results

Types of Tests	Maemo	Android
Smoketests	Ioana Chiorean PASS	Andreea - PASS
BFTs	Ioana Chiorean- PASS	Andreea & Nicolae (xti) - PASS
OS Integration	Ioana Chiorean PASS	Anna - PASS
L10n	mw22 - PASS	kbrosnan - PASS
Fixed Bug Verifications	All	All
Website and Device Stability Checks	--	kbrosnan - PASS
Updates	mw22 - PASS	--

Detailed Test Results

Smoketests Results

LG Optimus 2X [Andreea]

• Build: Mozilla /5.0 (Android;Linux armv7l;rv:2.1.1) Gecko/20110415 Firefox/4.0.2pre Fennec /4.0.1
• Results: PASS

• Notes:
  • No new issues found.
  • Known bugs:
    • bug 628269 - Pages are zoomed in at reload
    • bug 639504 - Unable to zoom in/out a webpage on header/footer when a double tap gesture is made

n900 [Ioana]

• Build: Mozilla /5.0 (Maemo; Linux armv7l;rv:2.1.1) Gecko/20110415 Firefox/4.0.2pre Fennec /4.0.1
• Results:PASS
• Notes:
  • Known issues:
    • bug 611294 - After sharing a page using Facebook, the browser returns to the shared page.
    • bug 639504 - Unable to zoom in/out a webpage on header/footer when a double tap gesture is made
    • bug 647220 - Sync data not cleared and added each sync time

BFT Results

LG Optimus 2X [Andreea]

• Build: Mozilla /5.0 (Android;Linux armv7l;rv:2.1.1) Gecko/20110415 Firefox/4.0.2pre Fennec /4.0.1
• Results: PASS

• Notes:
  • New issues found:
    • bug 650817 - Search engines list misinterprets tap and pan
    • bug 650820 - Adding more then 10 search engines removes the default ones

• • Options (Preferences):
    • Due to Bug 642718 feedback is removed from Options
  • General
    • Google instant blocks the "I'm Feeling Lucky" feature
    • known bugs:
      • bug 629645 - Strange behavior when using geolocation pages.
      • bug 628269 - Pages are zoomed in at reload
      • bug 639829 - Page position is not persistent on reload
  • Tabbed Browsing
    • known bugs:
      • bug 639794 - Opening a webpage from an external app should hide sidebars and panels
  • Downloading
    • known bugs:
      • bug 619054 - Inconsistency at auto-renaming
  • Layout
    • known bugs:
      • bug 580920 - Acid 2 test fails in Fennec
      • bug 614045 - Drop down menu items on http://www.bestbuymobile.com/ do not work
  • Add-ons Manager
    • The "Installing add-on" and "Add-on has been installed. Restart is required" dialogs appear in the Android notification bar not on the bottom right of the screen.

HTC Desire [Nicolae (xti)]

• Build: Mozilla /5.0 (Android;Linux armv7l;rv:2.1.1) Gecko/20110415 Firefox/4.0.2pre Fennec/4.0.1
• Results: PASS

• Notes:
  • New issues found:
    • bug 651048: Connec to Sync popup is still displayed even if the app is already connected to Sync
  • Form Assistant
    • known bugs:
      • bug 544614: Implement touch events
  • Context Menu
    • known bugs:
      • bug 641845: Long tap on a video which has a "poster" attribute will show an image context menu instead of a video context menu
  • Search
    • known bugs:
      • bug 641876: Popup messages overlap if two different search engine links are tapped
  • Security
    • known bugs:
      • bug 341604: Implement HTML5 sandbox attribute for IFRAMEs
      • bug 443564: Implement window.toStaticHTML
      • bug 446344: Implement Origin header CSRF mitigation
      • bug 471020: Add X-Content-Type-Options: nosniff support to Firefox
      • bug 528661: Heuristics to block reflected XSS
  • Web Usage
    • known bugs:
      • bug 486488: Highlighting objects doesn't work properly
      • bug 645042: Long tapping on an input field from a Javascript prompt box will open a context menu behind it
      • bug 647661: Fullscreen video controls are missing before the video starts to play
      • bug 544614: Implement touch events
  • System Pages
    • known bugs:
      • bug 641836: The search input should get the focus after loading about:config
  • Screen Navigation
    • known bugs:
      • bug 639504: Unable to zoom in/out a webpage on header/footer when a double tap gesture is made
      • bug 624254: Persistent checkerboarding when panning down on simple news page

n900 [Ioana Chiorean]

• Build: Mozilla /5.0 (Maemo; Linux armv7l;rv:2.1.1) Gecko/20110415 Firefox/4.0.2pre Fennec /4.0.1
• Results:In Progress
• Notes:
  • Known issues:
    • Options (Preferences):
      • Due to bug 642718 - there is no more feedback panel
      • bug 648543
      • Do Not Track- is not present
    • General
      • bug 639829 - Page position is not persistent on reload
      • bug 650820 - Adding more then 10 search engines removes the default ones
    • Tabbed Browsing
      • bug 639794 - Opening a webpage from an external app should hide sidebars and panels
    • Downloading
      • bug 619054 - Inconsistency at auto-renaming
      • bug 624691 - [Maemo-only] 'Open' button is disabled for downloaded files with .deb, .install and .pdf extension
    • Layout
      • bug 614045 - Drop down menu items on http://www.bestbuymobile.com/ do not work
    • Security
      • bug 341604 - (framesandbox) Implement HTML5 sandbox attribute for IFRAMEs
      • bug 443564 - Implement window.toStaticHTML
      • bug 446344 - Implement Origin header CSRF mitigation
      • bug 471020 - Add X-Content-Type-Options: nosniff support to Firefox
      • bug 528661 - Heuristics to block reflected XSS (like in IE8)
    • Context Menu
      • bug 641845 - Long tap on a video which has a "poster" attribute will show an image context menu instead of a video context menu
    • Web Usage
      • bug 544614 - Implement touch events
      • bug 486488 - Highlighting objects doesn't work properly on docs.google.com (here fandango.com)
      • bug 647661 - Fullscreen video controls are missing before the video starts to play

OS Integration Results

HTC Desire Z [Anna]

• Build: Mozilla /5.0 (Android;Linux armv7l;rv:2.1.1) Gecko/20110415 Firefox/4.0.2pre Fennec /4.0.
• Results: PASS
• Notes:
  • Known Bugs:
    • bug 617555: System menu button closes the VKB
  • Observations:
    • When the device is in landscape mode, using the Touch Input keyboard there are no "Next" and "Go" buttons
    • Couldn't install the swype keyboard (is not available on the android market)
    • Landscape full keyboard mode on / off works, using "widget.ime.android.landscape_fullscreen=0, widget.ime.android.fullscreen_threshold=300" is not implemented yet in Firefox 4.0.1

n900 [Ioana Chiorean]

• Build: Mozilla /5.0 (Maemo; Linux armv7l;rv:2.1.1) Gecko/20110415 Firefox/4.0.2pre Fennec /4.0.1
• Results: PASS
• Notes:
  • Known Issues:
    • bug 624691 - [Maemo-only] 'Open' button is disabled for downloaded files with .deb, .install and .pdf extension
    • bug 608340 - Maemo GTK : After dismissing the VKB once, you cannot bring up the VKB again by double clicking the awesome bar
    • bug 602882 - Directional pad up/down arrows not working in about:config
    • bug 624691 - [Maemo-only] 'Open' button is disabled for downloaded files with .deb, .install and .pdf extension

L10n Results

G2 [kbrosnan]

• Build: Build: Mozilla/5.0 (Maemo; Linux armv7l; rv:2.1.1) Gecko/20110415 Firefox/4.0.2pre Fennec/4.0.1
• Device: G2
• Results: PASS
• Notes:

n900 [mw22]

• Build: Mozilla/5.0 (Maemo; Linux armv7l; rv:2.1.1) Gecko/20110415 Firefox/4.0.2pre Fennec/4.0.1
• Results: PASS
• Notes: There is no Finnish locale for the multi build, so couldn't test that one.

Updates Results

n900 [mw22]

• Build: Mozilla/5.0 (Maemo; Linux armv7l; rv:2.1.1) Gecko/20110415 Firefox/4.0.2pre Fennec/4.0.1
• Results: PASS
• Notes: Note to self: always make sure you have the Ovi catalog disabled when doing updates testing on Maemo. Also make sure that no 'fennec' is there in the /var/lib/hildon-app-mgr/nokia-domain-system file. (added a notes section on https://wiki.mozilla.org/QA/Fennec/Updates#Maemo for that )