General Information

Area	Details
Team:	Developers: Designer: QA: Naoki Hirata

Test Plan

Introduction

This test plan is a test plan for testing the email app in the Gaia portion of b2g. This documentation will describe the testing strategy that will be used to validate the quality of the email app. Included are the various resources required to complete the testing.

Design Documentation/Background Research

Listed are the related documentation for testing:

Email Debugging info:

Module Owners

PM 1.0.1 Requirements:

https://docs.google.com/spreadsheet/ccc?key=0Ajt_meMlK3MOdHphdnV2a2RvcU9MZmd3cDlIQnZBMVE#gid=0

PM 1.1.0 Requirements:

https://docs.google.com/spreadsheet/ccc?key=0AtVT90hlMtdSdGw5QzNLZDczeWRFZ2JIOWJMUUZzSWc#gid=0

Specs:

Email Design Specs
1.0.1 Implementation is Activesync for Hotmail and IMAP for Gmail/Others

Automated Testing:

Repositories:

Tips and Tricks for testing:

Schedule Milestones

Scheduling is based on the milestone listed on the Release Management wiki for B2G Landings:

https://wiki.mozilla.org/Release_Management/B2G_Landing

Features that are required to land are listed here :

Leo Milestone

Lower priority bugs that may or may not make it into the milestones:

Leo milestone

Hardware Requirements

A desktop machine capable of running ADB is necessary for testing for getting debug information and flash the device.
Mobile devices listed
Optional: Build machine to build your own gaia/gecko

Mobile Device

Test devices :

Otoro
Unagi

Software Requirements

System Testing

Mobile OS

B2G - Firefox OS 18

Accounts

See Test Accounts

Email Servers

Hotmail.com
Gmail.com
Mozilla.com

Strategy

Smoke Tests are run daily for sanity check on builds
Exploratory Manual
- Modified Session Based testing around:
  - testing using outline; see Test Outline section
  - Bug fixes
  - Features implemented
Regression Testing based on Test Cases
- Found via bugs
- based on outline/feature/spec
Automation testing:
- Unit testing ( this is handled by developers)
- Other Preintegration testing such as Travis Continual Integration Testing (this is handled by developers)
- Marionette testing as covered in related documents
Performance Testing
- Perfomatic
- Note: we do need different specific email performance testing and there is a gap currently

Triage of bugs
- QA Response to Triaged bugs

Signoff Criteria

All email related leo+ blockers are closed
All smoke tests and basic functional tests are ran without finding any blockers
All leo+ blockers that are fixed are verified

Test Cases and Results

Test cases will be tracked in MozTrap under "gaia" and "email"

Test Outline

Note : This test out line is a brain storm of all possible conditions to consider for testing. The scope of the testing will be narrowed from this list based on design information, risks of breakage, etc.

Positive Smoke/Alpha Tests

UI

General

Gestures

Keyboard Short Cuts

Functional

UI Based Functionality
Input
- Symbols:
  - Astricks () {} [] & ^ % $ # @ ! ~ etc.
- Screen size (making sure that things fit)

Conditional Functionality
- Protocols:
  - IMAP:
    - Send
    - Receive
  - ActiveSync:
    - Send
    - Receive

Server

Beta Conditional

Virtual Conditions

Content Conditions

Physical Conditions
- While Charged
- While not Charged
- Power Consumption
- Performance
  - Speed of functions
  - Processor utilization
  - Memory utilization
    - Memory leaks?
    - Memory efficiency
- Storage
  - Disk
  - Sim

Negative (Tests to Break) Tests
- Security
  - Buffer Overflow (max characters)
- removal of characters

Race Conditions
- Lock conditions?
- tooltips or other dialogs?
- Network Conditions (Should have no impact)
  - WiFi
  - 3G
  - Bluetooth (Devices)
  - 4G?
  - SMS
- Signal Conflict
  - Browser notifications
    - clicking in search field (form assistant + virtual keyboard)
    - Restart notification (ie. change in certain preferences in preferences panel )
    - Password notification
    - form manager notification
    - password manager
    - popup blocker
    - offline storage
    - geolocation manager respective to content window
  - Battery
    - Battery Low
    - No Battery
  - Calls
    - Incoming Phone Call
  - Other Apps
    - Calendar notification
    - Alarm Clock notification
    - Sleep notification
    - Screensaver / Lock (system)
    - EnergySaver (Fade) (system)
    - Other?
    - Music playing (should not affect)
    - plugins (youtube and other websites with different media types)
    - Extensions ( addons to fennec)
- Buttons
  - Hard reset
  - Soft reset
  - Turn off phone
  - force quit
  - Other buttons being pressed on the phone (zoom, picture, power )
- UI State Change
  - Zoom
    - while bringing up Keyboard
    - while dismissing the keyboard
  - Change Orientation
    - while bringing up keyboard
    - while dismissing the keyboard

Device/Platform Specific Issues
- Portrait/Landscape Orientation
- Camera (should not conflict?)
- Microphone (Voice commands? Voice Recognition?)
- Speaker (should not conflict)
- headset (should not conflict)
- Vibrate mode (should not conflict)
- Volume (display conflict? need to test)
- GPS
- Print support?
- Sync
- Multiapp?
- physical keyboard
  - visible
  - hidden

(Note : There may be some difference between maemo and android)

Localization
- Double Byte Characters (Japanese, Chinese, Korean)
- RTL Languages (Arabic/Hebrew)
- High ASCII characters (Euro)
- Date Input/Format (Euro/Japanese/Chinese)
- Time Format (Chinese/Japanese)
- Dialog check
- Change of IME on Mobile OS

Help?

Usability

Usability
Accessibility (508 compliance)

B2G/QA/Gaia Test Plan/Email

Contents