MozillaWiki

B2G/QA/WebAPI Test Plan: Difference between revisions

Page Discussion

B2G/QA/WebAPI Test Plan (view source)

Revision as of 17:24, 30 March 2015

1,167 bytes added ,  30 March 2015
no edit summary
No edit summary
 
(47 intermediate revisions by 6 users not shown)
Line 1: Line 1:
=OBSOLETE=
Please note that this test plan is now obsolete, and is no longer underway.
=Boot2Gecko WebAPI Test Plan=
=Boot2Gecko WebAPI Test Plan=


Line 8: Line 12:
  |-
  |-
  | '''Contributors'''
  | '''Contributors'''
  | TBD
  | [mailto:mwargers@mozilla.com Martijn Wargers] (irc: mw22)<br />
[mailto:dclarke@mozilla.com David Clarke] (irc: onecyrenus)<br />
[mailto:jsmith@mozilla.com Jason Smith] (irc: jsmith)
  |-
  |-
  | '''Status'''
  | '''Status'''
  | Pre-release, approaching [https://docs.google.com/spreadsheet/ccc?key=0AiBigu584YY7dGlNSlY0QzhJb3M5anRBa1gxalV0Y3c#gid=0 Milestone 3]
  | Pre-release, approaching [https://docs.google.com/spreadsheet/ccc?key=0AiBigu584YY7dGlNSlY0QzhJb3M5anRBa1gxalV0Y3c#gid=0 Milestone 5]
|-
| '''Tracker'''
| [https://www.pivotaltracker.com/projects/578531 Pivotal Project]
  |-
  |-
  | '''Project Page'''
  | '''Project Page'''
  | [[WebAPI]]
  | [[WebAPI]]
|-
| '''References'''
| [[/Instructions for Contributors|Instructions for Contributors]]
  |}
  |}


Line 25: Line 37:
Functional test strategy will vary between APIs based on whether API effects can be effectively verified via automation, and whether that automation requires an emulator or not.
Functional test strategy will vary between APIs based on whether API effects can be effectively verified via automation, and whether that automation requires an emulator or not.


Broadly, testing will be divided between automation executed in continuous integration, and interactive exercises designed to manipulate APIs directly on-device. The expected amount of time given to each is roughly 75% to Planning and CI implementation and 25% to implementing interactive exercises. However, as the exercises will be necessary for final qualification of the B2G stack on-device, those priorities may change closer to the shipping date.
Broadly, testing will be divided between automation executed in continuous integration, and interactive exercises designed to manipulate APIs directly on-device. The expected amount of time given to each is roughly 75% to planning and CI implementation and 25% to implementing interactive exercises. However, as the exercises will be necessary for final qualification of the B2G stack on-device, those priorities may change closer to the shipping date.


=== CI-driven Automation ===
=== CI-driven Automation ===
Line 35: Line 47:


=== Interactive Exercises ===
=== Interactive Exercises ===
Some APIs have effects than cannot be verified through unattended automation. One example would be whether a microphone is open and receiving sound. Another example would be whether the phone is actually vibrating in response to an API call. These APIs must be verified interactively.
Some APIs have effects that cannot be verified through unattended automation. One example would be whether a microphone is open and receiving sound. Another example would be whether the phone is actually vibrating in response to an API call. These APIs must be verified interactively.


It would be tempting to use Gaia to exercise the API, but Gaia apps cannot be guaranteed to be using the APIs in a controlled fashion; that would require inappropriately locking in their implementation. So for these APIs, it is best to test using code specifically designed to exercise the APIs in a known fashion. In addition, as CI automation cannot be currently run on a final target device, it may be useful to have exercises even for APIs we address primarily through CI.
It would be tempting to use Gaia to exercise the API, but Gaia apps cannot be guaranteed to be using the APIs in a controlled fashion; that would require inappropriately locking in their implementation. So for these APIs, it is best to test using code specifically designed to exercise the APIs in a known fashion. In addition, as CI automation cannot be currently run on a final target device, it may be useful to have exercises even for APIs we address primarily through CI.
Line 42: Line 54:


Tentatively, the plan is to implement exercises as pages where possible for expediency's sake, but to eventually move them into a standalone app that developers and the testing community can experiment with.
Tentatively, the plan is to implement exercises as pages where possible for expediency's sake, but to eventually move them into a standalone app that developers and the testing community can experiment with.
Also see the [[/On_Device|On Device Test Plan]].


== API Sub-Plan List ==
== API Sub-Plan List ==
Line 47: Line 61:
{| class="fullwidth-table"
{| class="fullwidth-table"
  |-
  |-
  | style="width:28%" | '''API'''
  | style="width:20%" | '''API'''
  | style="width:54%" | '''Description'''
  | style="width:42%" | '''Description'''
  | style="width:18%" | '''Status'''
| style="width:20%" | '''API Status'''
  | style="width:18%" | '''Automation Status'''
  |-
  |-
  | [[/Screen Orientation|Screen Orientation]]
  | [[/Screen Orientation|Screen Orientation]]
  | Get notification when screen orientation changes as well as control which screen orientation a page/app wants.
  | Screen Orientation notifications and locking
  | [https://bugzilla.mozilla.org/show_bug.cgi?id=760735 In Progress]
| Ready
|-
  | [https://bugzilla.mozilla.org/show_bug.cgi?id=760735 First round done]
| [[/WebSMS|WebSMS]]
| Send/receive SMS messages as well as manage messages stored on device. 
| Planning
  |-
  |-
  | [[/Permissions|Permissions]]
  | [[/Web Activities|Web Activities]]
| Allow an app to manage all app permissions in a centralized location
  | Delegate an activity to another application.
| Not Started
  | Ready
|-
| Developer contacted
| [[/Contacts|Contacts]]
| Add/Read/Modify the device contacts address book.
| Not Started
|-
| [[/WebTelephony|WebTelephony]]
| Allow placing and answering phone calls as well as build in-call UI.
| Not Started
|-
| [[/Settings|Settings]]
| Set system-wide configurations that are saved permanently on the device.
| Not Started
|-
| [[/WebActivities|Web Activities]]
  | Delegate an activity to another application.  
  | Not Started
  |-
  |-
  | [[/Vibration|Vibration]]
  | [[/Vibration|Vibration]]
  | Control device vibration for things like haptic feedback in games.
  | Control device vibration for things like haptic feedback in games.
  | Not Started
  | Ready
| Developer contacted [https://bugzilla.mozilla.org/show_bug.cgi?id=679966 First round done by developer]
  |-
  |-
  | [[/Alarm|Alarm]]
  | [[/Alarm|Alarm]]
  | Schedule a notification, or for an application to be started, at a specific time.  
  | Schedule a notification, or for an application to be started, at a specific time.  
| Ready
| Tests in Suite
|-
| [[/Push Notifications|Push Notifications]]
| Allow the platform to send notification messages to specific applications.
| Unknown (TEF)
  | Not Started
  | Not Started
  |-
  |-
  | [[/Resource Lock|Resource Lock]]
  | [[/Geolocation|Geolocation]]
  | Prevent resources from being turned off, for example screen dimming, WiFi turning off, CPU going into sleep mode etc.  
  | Get current device position
| Ready
| Developer contacted
|-
| [[/Device Storage|Device Storage]]
| Add/Read/Modify files stored on a central location on the device.
| Ready
| First Round Completed, Blocked on Outstanding Bug
|-
| [[/Contacts|Contacts]]
| Add/Read/Modify the device contacts address book.
| Ready (Priv)
  | Not Started
  | Not Started
  |-
  |-
  | [[/Camera|Camera]]
  | [[/Camera|Camera]]
  | Control of front/rear cameras on-device
  | Control of front/rear cameras on-device
  | Not Started
  | Unknown
|-
| [[/WebBluetooth|Web Bluetooth]]
| Low level access to Bluetooth hardware. (Currently headset-only)
  | Not Started
  | Not Started
  |-
  |-
  | [[/TimeClock|Time/Clock]]
  | [[/TimeClock|Time/Clock]]
  | Set current time.
  | Get notifications about time changes (high priority). Set current time (low priority).
| Ready
  | Not Started
  | Not Started
  |-
  |-
  | [[Idle]]
  | [[Idle]]
  | Get notifications when user is idle.  
  | Get notifications when user is idle.  
| Ready
  | Not Started
  | Not Started
  |-
  |-
  | [[/BatteryStatus|Battery Status]]
  | [[/Battery Status|Battery Status]]
  | Information about battery charge level and if device is plugged in.  
  | Information about battery charge level and if device is plugged in.  
| Ready
  | Not Started
  | Not Started
  |-
  |-
  | [[/PushNotifications|Push Notifications]]
  | [[/FM Radio|FM Radio]]
  | Allow the platform to send notification messages to specific applications.
  | For FM radio feature.  
| In Progress (Priv)
  | Not Started
  | Not Started
  |-
  |-
  | [[/Mobile Connection|Mobile Connection]]
  | [[/WebSMS|WebSMS]]
  | Expose signal strength, operator, etc for GSM and other mobile connections. This does not cover WiFi.  
  | Send/receive SMS messages as well as manage messages stored on device.
| Ready (Priv)
| Planning
|-
| [[/Permissions|Permissions]]
| Allow an app to manage all app permissions in a centralized location
| In Progress (Priv)
  | Not Started
  | Not Started
  |-
  |-
  | [[PowerManagement|Power Management]]
  | [[/WebTelephony|WebTelephony]]
  | Turn on/off screen, cpu, device power, etc. Listen and inspect resource lock events.
  | Allow placing and answering phone calls as well as build in-call UI.  
| Ready (Priv)
  | Not Started
  | Not Started
  |-
  |-
  | [[/WiFi Info|Wifi Info]]
  | [[/Settings|Settings]]
  | Enumerate available WiFi networks, get signal strength and name of currently connected network, etc.  
  | Set system-wide configurations that are saved permanently on the device.  
| Ready (Priv)
  | Not Started
  | Not Started
  |-
  |-
  | [[/Geolocation|Geolocation]]
  | [[/Browser|Browser]]
  | Get current device position
  | Enables implementing a browser completely in web technologies.
| Ready (Priv)
| Not Started
|-
| [[/Web Bluetooth|Web Bluetooth]]
| Low level access to Bluetooth hardware. (Currently headset-only)
| Ready (Priv)
  | Not Started
  | Not Started
  |-
  |-
  | [[/FM Radio|FM Radio]]
  | [[/Mobile Connection|Mobile Connection]]
  | For FM radio feature.  
  | Expose signal strength, operator, etc for GSM and other mobile connections.
| Ready (Priv)
  | Not Started
  | Not Started
  |-
  |-
  | [[/Browser|Browser]]
| [[Power Management|Power Management]]
  | Enables implementing a browser completely in web technologies.  
| Turn on/off screen, cpu, device power, etc. Listen and inspect resource lock events.
| Ready (Priv)
| Tests in Suite
|-
  | [[/WiFi Info|Wifi Info]]
  | Enumerate WiFi networks, get signal strength and name of current network, etc.  
| Ready
  | Not Started
  | Not Started
|-
| [[/WebPayment|WebPayment]]
| Allow payments for virtual goods
| Ready
| Unit test automation present, Gaia UI Test automation currently being investigated
  |}
  |}
<small>Template for new Sub-Plans can be found [[B2G/QA/WebAPI Test Plan/Template|here]].</small>
Gmealer
canmove, Confirmed users
2,041

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/441872...1065209"