B2G/QA/WebAPI Test Plan
Boot2Gecko WebAPI Test Plan
Summary
| Lead | Geo Mealer (irc: geo) |
| Contributors | TBD |
| Status | Pre-release, approaching Milestone 3 |
| Project Page | WebAPI |
Scope
This test plan covers qualification of WebAPIs developed for or used specifically in Boot2Gecko to interface with mobile devices. It does not cover the entirety of Gecko APIs available. The individual APIs covered are listed below and broken out into their own individual sub-plans.
Currently, only functional testing is being planned. Ultimately, performance and security should be addressed. However, there are too many unknowns at time of writing to adequately plan for them in advance.
Strategy
Functional test strategy will vary between APIs based largely on how dependent they are on actual device hardware and whether API effects can be verified via automation. Broadly, testing will be divided between automation executed in continuous integration, and test apps designed to manipulate APIs directly on-device.
CI-driven Automation
B2G continuous integration will offer two options: mochitest-plain and a new Marionette-based JS harness.
Tests written against the new harness will run only in B2G CI against QEMU, a device emulator. They can set things like orientation or battery status to a known value, which allows for rich virtual device manipulation for fixture setup.
Tests written against mochitest-plain can run in the standard mozilla-central Tinderbox test environment and anywhere else mochitests are typically run, as well as against the B2G codebase. Mochitest will give wider coverage, but will not be able to use the emulator to set up device-related fixtures and is thus significantly more limited.
Test Apps
Some APIs have effects than cannot be verified through unattended automation. One example would be whether a microphone is open and receiving sound. Another example would be whether the phone is actually vibrating in response to an API call.
For these APIs, we must test using web apps designed to exercise the APIs in a known fashion such that the effects can be manually verified.
It would be tempting to use Gaia for this, but Gaia apps cannot be guaranteed to be using the APIs in a given way; they might refactor in the future. So, part of this testing project will be implementing apps whose API usage can be absolutely controlled.
It is still undetermined whether these apps will be implemented as web pages or installable apps. Apps would be closer to how a developer will mostly use these APIs. Pages don't require being preloaded onto the phone before testing can occur; they can just be navigated via onboard browser.
API Sub-Plan List
| API | Description | Status |
| Screen Orientation | Get notification when screen orientation changes as well as control which screen orientation a page/app wants. | In Progress |
| WebSMS | Send/receive SMS messages as well as manage messages stored on device. | Planning |
| Permissions | Allow an app to manage all app permissions in a centralized location | Not Started |
| Contacts | Add/Read/Modify the device contacts address book. | Not Started |
| WebTelephony | Allow placing and answering phone calls as well as build in-call UI. | Not Started |
| Settings | Set system-wide configurations that are saved permanently on the device. | Not Started |
| Web Activities | Delegate an activity to another application. | Not Started |
| Vibration | Control device vibration for things like haptic feedback in games. | Not Started |
| Alarm | Schedule a notification, or for an application to be started, at a specific time. | Not Started |
| Resource Lock | Prevent resources from being turned off, for example screen dimming, WiFi turning off, CPU going into sleep mode etc. | Not Started |
| Camera | Control of front/rear cameras on-device | Not Started |
| Web Bluetooth | Low level access to Bluetooth hardware. (Currently headset-only) | Not Started |
| Time/Clock | Set current time. | Not Started |
| Idle | Get notifications when user is idle. | Not Started |
| Battery Status | Information about battery charge level and if device is plugged in. | Not Started |
| Push Notifications | Allow the platform to send notification messages to specific applications. | Not Started |
| Mobile Connection | Expose signal strength, operator, etc for GSM and other mobile connections. This does not cover WiFi. | Not Started |
| Power Management | Turn on/off screen, cpu, device power, etc. Listen and inspect resource lock events. | Not Started |
| Wifi Info | Enumerate available WiFi networks, get signal strength and name of currently connected network, etc. | Not Started |
| Geolocation | Get current device position | Not Started |
| FM Radio | For FM radio feature. | Not Started |
| Browser | Enables implementing a browser completely in web technologies. | Not Started |