Fennec/Features/TabletUI/TestPlan

Test Plan

Introduction

Design Documentation/Background Research

The purpose of the virtual keyboard is to provide an interface to fennec for people to type in various languages when a hardware keyboard is closed or not available to the device.

Description of this Document

This document will cover the testing approach for virtual keyboards on the Tablets. This document will not cover the testing approach for smaller handheld device, but may be using most of the same techniques for testing VKB for smaller handheld devices. There are some notable differences, such as the virtual keyboard is not full screen for landscape view.

Related Documents

See Virtual Keyboard for Android Test Plan

Schedule and Milestones

See Feature Page

Resource Requirements

Hardware

Test devices: Handhelds for comparison:

• Motorola Droid 2
• HTC Thunderbolt

Tablets:

• HTC Flyer
• Samsung

System Testing

• Android OS 2.2 (Froyo)
• Android OS 2.3 (Gingerbread)
• Android OS 3.1 (Honeycomb)

Software

Variety of VKB/IMEs  : see...

Staffing

Feature Lead

Responsibilities

• Testing Devices
• Training other people in VKBs when asked
• Verifying unconfirmed bugs
• triaging bugs
• verifying fixes

Training

• none necessary

Note:

1. need to know how to install VKBs
2. need to know how the VKB works

Features To Be Tested / Test Approach

New Features Testing

The VKB has a slightly different behavior on the Tablet platform. The differences are listed within the bug list and the ui design described in the main feature page
The test outline below will show the items to be tested.

Test Deliverables

• Bug Reports
• Bug verifications
• Litmus Test cases

Dependencies/Risks

• Change in theme may alter some of the ways to bring up the keybaord
• VKBs are generally third party, and some crashes may be caused by the interaction between the VKB and Fennec
• VKB bugs are not the same as IME bugs and vice versa; knowing the differences between the two can be hard without a developer looking into it.

Entrance/Exit Criteria

• The top 3 keyboards, an Asian keyboard (winCE?) and European Keyboard does not have any crashers, hangers, etc.

Test Outline

Note :

1. This test out line is a brain storm of all possible conditions to consider for testing. The scope of the testing will be narrowed from this list based on design information, risks of breakage, etc.
2. When testing, each item has to be checked, for performance, visual impact (ie graphic defects), race conditions, locking, etc. Some things will be called out to have specific instructions to check for, however the testing should not be limited to only the called out item.

Positive Smoke/Alpha Tests

UI

1. General

Keyboard Menu

• hardware menu button used in various portions of the application

Note: the home and back buttons are special. Back button has a history of leading to a lot of graphic defects current

Copy

• copy one character, see what's in clipboard
• copy more than one character, see what's in the clipboard
• copy all characters, see what's in the clipboard
• copy high ascii/double byte characters to see if there's any corruption in characters when pasting

Note: Generally with copying you want to verify that the proper thing is being copied from the selection line.

Cut

• cut one character, see what's in the clipboard
• cut more than one character, see what's in the clipboard
• cut all characters, see what's in the clipboard
  • hit delete after cutting all characters, and typing
• cut high ascii/double byte characters to see if there's any corruption in characters when pasting

Note: Generally with cutting you want to verify that the proper thing is being copied from the selection line;
there should be no graphic defect when typing (ie no typing occurs, no smudges after cutting, etc)

Paste

• From the same application, generally after cutting or copying, paste will help see what's in the clipboard
• From another Application
  • copy from another application and pasting
  • cutting from another application and pasting

Note: Generally with pasting you want to verify that the proper thing is being pasted into the selection line. Usually double byte characters (different formats) and high ascii (compound accented characters) have a tendency to be corrupt.

Highlights

• select one character and pan
  • select more than one character and pan
  • select all characters and pan
  • do the above type selections and use copy
  • do the above type of selections and use cut
• cursor insertions
  • select using the cursor versus shift arrows

Note: positioning is important, you may be looking at if the cursor deletes at the proper location, or adds any graphic defects when typing/pasting or removing characters

Touch Testing

• Single Touch
  • Long Touch
  • short touch
  • Single key Press : depends on IME and key
  • Special Key : See Functionality
• Double Touch
  • Two keys pressed : depends on IME and key
• Triple+ Touch (negative test case)
  • Three keys pressed : depends on IME and key
  • four keys pressed : depends on IME and key

Note: triple touch or more should not do anything, these are just negative test cases to verify that the single touch/double touches do exactly as they are suppose to do

Keyboard Short Cuts

• support Short Cuts
• Dev Cheat Codes

Note: Some keyboard shortcuts do not translate to certain devices due to the lack of certain hardware keys. Typically control characters should be the most tested as they have a history of causing odd behavior. long tap may cause keys to remain on screen if typing fast in most other portions or dismissing keyboards, fast short typing may cause some issues with text appearing.

Functional

1. UI Based Functionality

Keyboard Functionality

• verify that the keys function accordingly
• special keys :
  • White Space
    • Space, Enter, Tab Return
  • Modifier Keys
    • Shift, Caps, Option, Command, Control
  • Positional Keys :
    • Arrows, Home, End, PageUp/Down
  • Symbols:
    • Astricks () {} [] & ^ % $ # @ ! ~ etc.
  • Other:
    • Esc, Delete, Clear
    • key combos
    • shortcuts
• Screen size (making sure that things fit)

Note: Generally Menu+VKB typically does not do anything. There are oddities to watch for such as hitting shift, letting go and then typing multiple characters should only capitalize one character. Special characters may end up doing something different due to encoding, using web symbols and copy paste, etc. White spacings may disappear due to copy/paste etc. It's more or less doing the same testing as the above but looking at the functionality of the keys versus the functionality of the actions. You can run both tests at the same time provided you observe for these functionalities and the above.

2. Conditional Functionality

Selections/Show/Hide/Drag/Drop Keyboard

• dismiss keyboard
  • click on space above keyboard
  • hit return?
• start virtual keyboard
  • click in a text field selection
  • click in a search field
  • click in awesome bar
• selection
  • drag/drop
  • character selection
  • multicharacter selection
  • Word selection
  • multiword selection
  • split word selection
    • hyphenated
    • carriage return
    • partial character in words

Note: One path of closing/opening the VKB may not necessarily lead to the same state as another. Timing can change things, be careful to check for various actions that could cause issues with graphic defects. Esp with the system back button

Beta Conditional

Virtual Conditions

• click in text box
• double click in text box
• click outside of Virtual Keyboard : dismissing keyboard
• scrolling above the on-screen keyboard
• Change Cursor Focus
  • change focus after composition
• slide right, slide left [ Panels ]
• AwesomePage
• Preferences/Feedback/Plugins/etc Panels
• Context Menus
• Site Menu
• Favorites Panel
• Position/Graphical issues
  • collapsing before bringing up keyboard/dismissing keyboard
  • scrolling before bringing up keyboard/dismissing keyboard
  • tapping after dismissing the keyboard should bring the keyboard up

Content Conditions

• html4
• html5
• asp

HW keys

• Back button
• menu button

Physical Conditions

• While Charged
• While not Charged
• Power Consumption
• Performance
  • Speed of functions
  • Processor utilization
  • Memory utilization
    • Memory leaks?
    • Memory efficiency
• Storage
  • Disk
  • Sim

Negative (Tests to Break) Tests

• Security
  • Buffer Overflow (max characters)
• removal of characters

Race Conditions

• Lock conditions?
• tooltips or other dialogs?
• Network Conditions (Should have no impact)
  • WiFi
  • 3G
  • Bluetooth (Devices)
  • 4G?
  • SMS
• Signal Conflict
  • Browser notifications
    • clicking in search field (form assistant + virtual keyboard)
    • Restart notification (ie. change in certain preferences in preferences panel )
    • Password notification
    • form manager notification
    • password manager
    • popup blocker
    • offline storage
    • geolocation manager respective to content window
  • Battery
    • Battery Low
    • No Battery
  • Calls
    • Incoming Phone Call
  • Other Apps
    • Calendar notification
    • Alarm Clock notification
    • Sleep notification
    • Screensaver / Lock (system)
    • EnergySaver (Fade) (system)
    • Other?
    • Music playing (should not affect)
    • plugins (youtube and other websites with different media types)
    • Extensions ( addons to fennec)
• Buttons
  • Hard reset
  • Soft reset
  • Turn off phone
  • force quit
  • Other buttons being pressed on the phone (zoom, picture, power )
• UI State Change
  • Zoom
    • while bringing up Keyboard
    • while dismissing the keyboard
  • Change Orientation
    • while bringing up keyboard
    • while dismissing the keyboard

Device/Platform Specific Issues

• Portrait/Landscape Orientation
• Camera/scanner VKBs
• Microphone (Voice commands? Voice Recognition?)
• Speaker (should not conflict)
• headset (should not conflict)
• Vibrate mode (should not conflict)
• Volume (display conflict? need to test)
• GPS
• Print support?
• Sync
• Multiapp?
• physical keyboard
  • visible
  • hidden

Localization

• Double Byte Characters (Japanese, Chinese, Korean)
• RTL Languages (Arabic/Hebrew)
• High ASCII characters (Euro)
• Date Input/Format (Euro/Japanese/Chinese)
• Time Format (Chinese/Japanese)
• Dialog check
• Change of IME on Mobile OS

Help

• context sensitive help?

Usability

Usability

• Keyboard Real Estate
• scrolling away?
• jumping to another field?
• Encryption
• Keyboard size?
• Fullscreen/Halfscreen?
• Workflow?

Accessibility (508 compliance)

• Voice Commands?
• Scanner/VKB?