View source for B2G/QA/Device Test Plan/NewDevice


= Test Plan =
== Introduction ==
This test plan is a test plan for testing new devices and some of the things to look out for.
This is a guideline; there may be more to test depending on the feature of the phone.

=== Design Documentation/Background Research ===
Listed are the related documentation for testing:<br>
==== Module Owners ====
* [[Modules/FirefoxOS|List of Firefox OS Modules/Dev Owners (link)]] - in case you need to ask questions regarding the new features of the component.<br>
* [[Firefox_OS/functionalteams|'''Firefox OS functional teams breakdown (link)''']]<br>
* [[FirefoxOS/FeatureTesting|'''QA owners for Firefox OS components (link)''']]

====  Requirements: ====
* [https://wiki.mozilla.org/Firefox_OS/Flame Flame]
* [[B2G/QA/Woodduck_tracking|Woodduck Wiki]]
* [https://wiki.mozilla.org/Firefox_OS/Tarako Tarako]
* [https://mana.mozilla.org/wiki/display/PM/Sony Tako]
* [https://mana.mozilla.org/wiki/display/PM/Project+Dolphin Dolphin]
* [https://wiki.mozilla.org/Firefox_OS/Smart_TV Smart TV]

==== Bugs tracking features: ====
* [http://charts.mozilla.org/ Dashboard - High Priority Bugs (Overall)]
* [https://wiki.mozilla.org/Firefox_OS/OneDotTwoTracking FxOS 1.2 Dashboard]
* [https://wiki.mozilla.org/Firefox_OS/OneDotThreeTracking/Dashboard FxOS 1.3 Dashboard]
* [https://wiki.mozilla.org/Firefox_OS/TwoDotTwoTracking/Dashboard FxOS 2.2 Dashboard]

==== Specs: ====
* See [https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/Phone_guide/Phone_specs Device Specifications]

==== Tips and Tricks for testing: ====
* https://wiki.mozilla.org/B2G/QA/Tips_And_Tricks
* https://developer.mozilla.org/en-US/docs/Performance/Profiling_with_the_Built-in_Profiler

==== OS Diagrams ====
* https://github.com/sotaroikeda/firefox-diagrams/wiki/Firefox-Diagrams

=== Schedule Milestones ===
== Hardware Requirements ==
* A desktop machine capable of running ADB is necessary for testing for getting debug information and flash the device.
* Mobile devices listed
* Bluetooth headset
* headset with microphone
* headset without microphone
* another device/machine with bluetooth transfer capabilities
* Optional: Build machine to build your own [https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/Firefox_OS_build_prerequisites?redirectlocale=en-US&redirectslug=Firefox_OS%2FFirefox_OS_build_prerequisites gaia/gecko]
* Optional: Build OS Storage machine to make it easier to find regressions

=== Mobile Device ===
* New Device

== Software Requirements ==
* [http://developer.android.com/sdk/index.html ADB]
* fastboot
* New Device Flashing Tool and USB Driver
* System for Flashing Tool (Typically a Windows Machine)

== System Testing ==
=== Mobile OS  ===
* Depends on device

=== Accounts ===
* See [https://intranet.mozilla.org/QA#Test_Accounts Test Accounts]
* Facebook Contact Import accounts also listed in the above link

=== Email Servers ===
* Hotmail.com
* Gmail.com
* Mozilla.com

== Strategy ==
* smoke test testing
* target testing for interface between hardware and software
* Exploratory testing

Currently not tested:
* Performance against different mobile data types

== Test Deliverables ==
* Test Results

== Dependencies/Risk ==
The drivers on most devices are given from the OEM vendors.  This poses a risk if we do not get an update to a potentially critical bug.

== Signoff/Exit Criteria ==
* no critical bug is found during testing

= Test Outline =
Note : This test out line is a brain storm of all possible conditions to consider for testing.  The scope of the testing will be narrowed from this list based on design information, risks of breakage, etc.

== Positive Smoke/Alpha Tests ==

=== Functional ===
==== Smoke Test ====
===== Email =====
* Test email on wifi and mobile data
===== Gallery =====
* test various images and various image size (supported/unsupported)
===== Camera =====
* test orientation
* test video capture
* test camera/camera preview
* flash
* front camera/back camera
===== FM Radio =====
* test audio for stereo output
===== Music =====
* music (supported/unsupported encoding)
===== Video =====
* video  (supported/unsupported encoding)
* streaming video (youtube)
===== Usage =====
* Usage Graph
===== Geolocation =====
* GPS test app or maps.google.com
===== SIM =====
====== Dialer ======
* make a call in
* make a call out
* conference call
* call waiting
* dual sim support
** 0 sim, 1 sim, 2 sim
* sim lock
* airplane mode
* sms
====== tethering ======
* tethering
====== Wifi Hotspot ======
* wifi hotspot
===== touch screen =====
* sensitivity
===== bluetooth =====
* file transfer
* music control
* dialer
===== NFC =====
===== Buttons & Misc =====
* power button
* volume buttons
* Vibration
* headset
* microphone
* orientation/angle

===== OTA =====
* check wifi
* check bluetooth 
* check NFC
* check camera
** front and back
** check flash
** check autofocus
* check music
* check geolocation
* check dialer
* check SMS
* check USB storage
* check FM Radio
* check email
* check logshake
* check light sensor ( ie turn on auto brightness and place finger over light sensor )

== Beta Conditional / Exploratory Items ==
*OTA
** Shallow flashing
*** testing full
** changing the channel to nightly/aurora, updating
*** testing full
*Physical Conditions 
**Power Consumption
**Performance
*** Speed of functions
*** Processor utilization
*** Memory utilization
**** Memory leaks?
**** Memory efficiency
**Storage
***Disk
***Sim
*Race Conditions 
** Lock conditions?
** tooltips or other dialogs?
** Network Conditions (Should have no impact)
*** WiFi (various supported ranges)
*** Bluetooth (Devices)
*** Mobile Data type : 
**** 3G/4G?
**** signal frequency
*** SMS
**Signal Conflict 
**Battery 
****Battery Low 
****No Battery 
****While Charged 
****While not Charged
***Other Apps 
****Calendar notification 
****Alarm Clock notification 
****Sleep notification
****Screensaver / Lock (system) 
****EnergySaver (Fade) (system) 
****Other? 
****Music playing (should not affect)
****plugins (youtube and other websites with different media types)
****Extensions ( addons to fennec)
**Buttons 
***Hard reset 
***Soft reset 
***Turn off phone 
***force quit 
***Other buttons being pressed on the phone (zoom, picture, power ) 
** Endurance Testing

== Usability ==
*Usability
* Accessibility (508 compliance)