QA/Fennec/Milestones/5.0b4

« Fennec:fx5b3 | Fennec:fx5b4 | Fennec:fx5b5 »

Fennec 5.0 Beta Test Plan

• Current Milestone: Firefox mobile 5
• Release: June 7, 2011
• QA Contact: Kevin Brosnan
• IRC: irc://irc.mozilla.org#qa :kbrosnan
• Builds: Firefox 5.0b4 | changeset | pushlog

Description

This is the Firefox mobile beta release test plan .

• Firefox mobile smoketests
• Firefox mobile basic functional tests (BFT)
• Feature exploratory testing
• Validating fixes
• L10N spot checks
• Website signoff

Current Cycle

The current cycle for Firefox mobile beta is focused on stabilizing for release and verifying fixes that will ship with Firefox 5.

Test Results

Test results:

Bugs found:

• Track using [ [fx5mb]] in the whiteboard

Milestone: Firefox mobile 5.0b4

Builds: ftp://ftp.mozilla.org/pub/mobile/candidates/5.0b4-candidates/

Known Issues: The following issues are already known and should not be reported again

Smoketests

Smoketests cover broad areas of testing for Firefox mobile. These represent core functions of the browser. Any failures during the smoketest should be addressed before continuing with the other testing.

Android

Results:

• Device: Samsung Google Nexus S (Android 2.3)
• Build: Mozilla /5.0 (Android;Linux armv7l;rv:5.0) Gecko/20110527 Firefox/5.0 Fennec/5.0
• Results Fennec 5.0b3 build 1 : PASS

Known bugs:

• bug 628269 - Pages are zoomed in at reload
• bug 639504 - Unable to zoom in/out a webpage on header/footer when a double tap gesture is made

Note :

• no update test made - updates will be made only through the Android Market.

Maemo

• Device: Nokia N900 (Maemo GTK)
• Build: Mozilla/5.0 (Maemo; Linux armv7l; rv:5.0) Gecko/20110527 Firefox/5.0 Fennec/5.0
• Results Fennec 5.0b4 build 1 : PASS

Known bugs:

• bug 661098: After sharing a webpage using facebook, the "Share link page" does not close
• bug 661111: [Maemo] No JPAKE authentication code. Failed to connect to the server error

Note :

• Firefox Beta could not retrieve add-on. I can only install add-ons from AMO if "extensions.checkCompatibility.5.0=false"

BFT

Basic functional tests cover specific features and ensure continued quality.

• Assigned: Anna & Andreea
• Device: HTC Desire Z (Android 2.2) & LG Optimus 2X (Android 2.2)
• Build: Mozilla /5.0 (Android;Linux armv7l;rv:5.0) Gecko/20110603 Firefox/5.0 Fennec/5.0
• Results:PASS
  • General
    • known bugs:
      • bug 628269: Pages are zoomed in at reload
      • bug 639829: Page position is not persistent on reload
      • bug 629645: Strange behavior when using geolocation pages.
  • Tabbed Browsing
    • known bugs:
      • bug 639794: Opening a webpage from an external app should hide sidebars and panels
  • Bookmarks
    • known bugs:
      • bug 642635: Adding/removing a bookmark isn't reflected in the awesomebar results popup
  • Downloading
    • known bugs:
      • bug 619054: Inconsistency at auto-renaming
  • Sync
    • known bugs:
      • {bug 656594: Sync Now button is active even if the Sync is in progress
  • Layout
    • known bugs:
      • bug 580920: Acid 2 test fails in Fennec (happens only in portrait mode)
      • bug 614045: Drop down menu items on http://www.bestbuymobile.com/ do not work
  • Form Assistant:
    • known bugs:
      • bug 652831: Drop down menu flickers on Bugzilla on text fields
  • Context Menu:
    • known bugs:
      • bug 641845: Long tap on a video which has a "poster" attribute will show an image context menu instead of a video context menu
  • Security
    • known bugs:
      • bug 341604: (framesandbox) Implement HTML5 sandbox attribute for IFRAME
      • bug 443564: Implement window.toStaticHTML
      • bug 446344: Implement Origin header CSRF mitigation
      • bug 471020: Add X-Content-Type-Options: nosniff support to Firefox
      • bug 528661: Heuristics to block reflected XSS (like in IE8)
  • System pages:
    • known bugs:
      • The search field does not get focus, so you can directly type something after about:config has loaded due to bug 641836 whose target milestone is Firefox 6.
  • Web Usage:
    • known bugs:
      • bug 652831: Drop down menu flickers on Bugzilla on text fields.(See Comment 5)
  • Screen Navigation
    • Known bugs:
      • bug 639504: Unable to zoom in/out a webpage on header/footer when a double tap gesture is made.
      • bug 624254: Persistent checkerboarding when panning down on simple news page
      • bug 658234: Panning outside of an Iframe causes the content of the frame to scroll
  • RTL
    • Known bugs
      • Search engines don't appear but this is expected due to bug 645970, tracked only in fennec 6.

Features Exploratory Testing

Identify the feature, description the testing you performed, the device you tested on, and any bugs filed

Sync (kbrosnan)

• Device Tested: N1, G2, Ideos S7
• Testing Performed: Tested on Android 2.2 and 2.1. Verified that on large sync profiles Firefox 5 beta 3 showed the issue and that Firefox 5 beta 4. Tested that upgrading a broken sync to Firefox 5 beta 4 repairs broken sqlite-wal files. Tested that the fix works when Firefox mobile is installed to system storage or when moved to the sdcard.
• Issues Found:
  • None found

Bug Fixes

Verify the bug fixes which landed for this beta cycle.
bug 656782 - Runtime CPU features detection does not work for pixman in Android
bug 660036 - places.sqlite-wal size grows more in Firefox 5 than in Firefox 4
Additional changes that are picked up from Core/Toolkit can be found at the mobile pushlog.

Issues Found/Reopened Bugs

• None found

L10N Spot Checks

Light testing on featured locales to ensure strings fit the space allotted and there are no entity errors from missing or corrupt localizations.

• kbrosnan - Done

Website Sign-off

Review website copy for error and suggest items that should make the release notes/known issues