Firefox OS/FeatureTesting: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
 
(14 intermediate revisions by the same user not shown)
Line 104: Line 104:


|-
|-
| QA || TBD
| QA || [mailto:whsu@mozilla.com William Hsu]
|-
|-
| How to test || Please refer to MozTrap
| How to test || Please refer to MozTrap
Line 237: Line 237:


== System frontend ==
== System frontend ==
* Market Customization Framework
{| class="wikitable"
{| class="wikitable"
|-
|-
| Feature|| Homescreen, App-grouping
| Feature|| Homescreen, App-grouping
|-
|-
| QA || [mailto:hcheng@mozilla.com Hermes Cheng] [mailto:nhirata@mozilla.com Naoki Hirata]
| QA || [mailto:nhirata@mozilla.com Naoki Hirata]
|-
|-
| How to test ||  
| How to test || [https://goo.gl/a14Uuz Homescreen test specification]
|-
|-
| Moztrap || [https://moztrap.mozilla.org/manage/suites/?pagenumber=1&pagesize=20&sortfield=created_on&sortdirection=desc&filter-name=%5Bsys+fe%5D+vertical+homescreen [Sys FE<nowiki>]</nowiki> Vertical Homescreen]
| Moztrap || [https://moztrap.mozilla.org/manage/suites/?pagenumber=1&pagesize=20&sortfield=created_on&sortdirection=desc&filter-name=%5Bsys+fe%5D+vertical+homescreen [Sys FE<nowiki>]</nowiki> Vertical Homescreen]
Line 250: Line 249:
| Note ||
| Note ||
|}
|}
* Notifications
 
* Download Manager
{| class="wikitable"
* Browser
|-
* App/Datastore Permissions
| Feature|| Browser
* First Time Experience
|-
* E.me Integration
| QA || [mailto:ashiue@mozilla.com Alison Shiue]
* Offline Error Handling
|-
| How to test || [https://goo.gl/lmsOi0 Browser test specification]
|-
| Moztrap || [https://goo.gl/e9KwQb Browser test cases]
|-
| Note ||
|}
 
{| class="wikitable"
|-
| Feature|| Notifications
|-
| QA || NA
|-
| How to test || TBD
|-
| Moztrap || [https://goo.gl/29Tcg6 Notifications test cases]
|-
| Note ||
|}
 
{| class="wikitable"
|-
| Feature|| First Time Experience
|-
| QA || [mailto:ashiue@mozilla.com Alison Shiue]
|-
| How to test || [https://goo.gl/XRVw6T FTE test specification]
|-
| Moztrap || [https://goo.gl/BPkqdm FTE test cases]
|-
| Note ||
|}
 
{| class="wikitable"
{| class="wikitable"
|-
|-
| Feature|| Task Manager
| Feature|| Task Manager
|-
|-
| QA || [mailto:hcheng@mozilla.com Hermes Cheng][mailto:nhirata@mozilla.com Naoki Hirata]
| QA || [mailto:nhirata@mozilla.com Naoki Hirata]
|-
|-
| How to test ||  
| How to test ||  
Line 269: Line 301:
| Note ||
| Note ||
|}
|}
* Soft homebutton
 
* Find My Device
{| class="wikitable"
|-
| Feature|| Download Manager
|-
| QA || TBD
|-
| How to test || Please refer to MozTrap
|-
| Moztrap || [https://goo.gl/11QN7D Download manager test cases]
|-
| Note ||
|}
 
{| class="wikitable"
|-
| Feature|| Find My Device
|-
| QA || TBD
|-
| How to test || Please refer to MozTrap
|-
| Moztrap || [https://goo.gl/5AY1qg Find My Device test cases]
|-
| Note ||
|}
 
{| class="wikitable"
{| class="wikitable"
|-
|-
| Feature|| Rocketbar, Search App
| Feature|| Rocketbar, Search App
|-
|-
| QA || [mailto:hcheng@mozilla.com Hermes Cheng][mailto:nhirata@mozilla.com Naoki Hirata]
| QA || [mailto:nhirata@mozilla.com Naoki Hirata]
|-
|-
| How to test ||  
| How to test || [https://goo.gl/ae1PG3 Rocketbar]
|-
|-
| Moztrap || [https://moztrap.mozilla.org/manage/suites/?pagenumber=1&pagesize=20&sortfield=created_on&sortdirection=desc&filter-name=%5Bgeneral%5Drocketbar [General<nowiki>]</nowiki>Rocketbar]
| Moztrap || [https://moztrap.mozilla.org/manage/suites/?pagenumber=1&pagesize=20&sortfield=created_on&sortdirection=desc&filter-name=%5Bgeneral%5Drocketbar [General<nowiki>]</nowiki>Rocketbar]
Line 283: Line 340:
| Note ||
| Note ||
|}
|}
{| class="wikitable"
|-
| Feature || Market Customization
|-
| QA || TBC
|-
| How to test || You can customize Gaia's default settings, like with a different wallpaper or with support phone already stored inside Contacts, or customize by carrier's MNC, MCC, so the settings can be different if you insert a specific SIM card. See [[B2G/QA/Customizations|wiki]] and [https://developer.mozilla.org/en-US/Firefox_OS/Developing_Gaia/Market_customizations_guide mdn] for more details.
|-
| Moztrap || [https://moztrap.mozilla.org/manage/suites/?pagenumber=1&pagesize=20&sortfield=created_on&sortdirection=desc&filter-name=customized Customized UI Suite] 
|-
| Note || Supported since Firefox OS v1.X
|}
* App/Datastore Permissions
* E.me Integration
* Offline Error Handling
* Soft home button
* Panning and Zooming
* Panning and Zooming


== Telephony ==
== Telephony ==
* Multi Sim
* IPV6
* LTE
{| class="wikitable"
{| class="wikitable"
|-
|-
| Module || RIL
| Module || RIL
|-
|-
| QA || [mailto:hcheng@mozilla.com Hermes Cheng]
| QA || [mailto:gchang@mozilla.com Gerry Chang]
|-
|-
| How to test || Follow the guide to [https://wiki.mozilla.org/FirefoxOS/DeviceQA/CollectLogs#RIL_Debugging turn on RIL flag for adb debug log ]  
| How to test || Follow the guide to [https://wiki.mozilla.org/FirefoxOS/DeviceQA/CollectLogs#RIL_Debugging turn on RIL flag for adb debug log ]  
|-
|-
| Moztrap ||  
| Moztrap || [https://goo.gl/DKyczb RIL test cases]
|-
|-
| Note ||  
| Note ||  
Line 305: Line 375:
{| class="wikitable"
{| class="wikitable"
|-
|-
| Module || CDMA
| Module || Cellbroadcast, CMAS
|-
|-
| QA || [mailto:hcheng@mozilla.com Hermes Cheng]
| QA || TBC
|-
|-
| How to test || Wasabi is the current CDMA phone we have, full flash, and follow the [https://intranet.mozilla.org/QA/B2G_Tips_and_Tricks#Wasabi instruction] to hard code the networkType=4 in ril_workers.js
| How to test || Currently they can be tested using emulator.
|-
|-
| Moztrap ||  
| Moztrap ||  
[https://moztrap.mozilla.org/manage/cases/?pagenumber=1&pagesize=20&sortfield=created_on&sortdirection=desc&filter-tag=2985 Cell broadcast] All test cases
|-
|-
| Note ||  
| Note ||  
|}
|}


== Connectivity ==
{| class="wikitable"
{| class="wikitable"
|-
|-
| Module || STK
| Module || RTSP
|-
|-
| QA || [mailto:hcheng@mozilla.com Hermes Cheng]
| QA || [mailto:whsu@mozilla.com William Hsu]
|-
|-
| How to test ||  
| How to test || Please refer to MozTrap
Use this script to extract the adb log: https://gist.github.com/shinglyu/e44502cbb78e6e207445
|-
|-
| Moztrap ||  
| Moztrap || [https://goo.gl/F1Bj2H RTSP test cases]
[https://moztrap.mozilla.org/manage/suites/?pagenumber=1&pagesize=20&sortfield=created_on&sortdirection=desc&filter-name=simtoolkit SimToolKit] All test cases
|-
|-
| Note ||  
| Note ||  
Line 333: Line 403:
{| class="wikitable"
{| class="wikitable"
|-
|-
| Module || Cellbroadcast, CMAS
| Module || NFC
|-
|-
| QA || [mailto:hcheng@mozilla.com Hermes Cheng]
| QA || [mailto:ashiue@mozilla.com Alison Shiue]
|-
|-
| How to test || Currently they can be tested using emulator.. T
| How to test || [https://goo.gl/Cxt3vt NFC test specification]
|-
|-
| Moztrap ||  
| Moztrap || [https://goo.gl/cwULzU NFC test cases]
[https://moztrap.mozilla.org/manage/cases/?pagenumber=1&pagesize=20&sortfield=created_on&sortdirection=desc&filter-tag=2985 Cell broadcast] All test cases
|-
|-
| Note ||  
| Note ||  
|}
|}


== Connectivity ==
{| class="wikitable"
* RTSP
|-
* NFC
| Module || WIFI
* WIFI
|-
| QA || [mailto:gchang@mozilla.com Gerry Chang]
|-
| How to test || Please refer to MozTrap
|-
| Moztrap || [https://goo.gl/KDsTPG WIFI test cases]
|-
| Note ||
|}
* WIFI Direct
* WIFI Direct
* WIFI Display
* WIFI Display
Line 376: Line 453:
|}
|}


* SD Card formatting
* MTP
* Metrics testing (FTU, APP usage)
{| class="wikitable"
{| class="wikitable"
|-
|-
Line 386: Line 460:
|-
|-
| How to test ||  
| How to test ||  
FTU Ping:  https://wiki.mozilla.org/B2G/QA/Ping_Testing     
[https://wiki.mozilla.org/B2G/QA/Ping_Testing FTU Ping]    


App Usage: https://wiki.mozilla.org/B2G/QA/Ping_Testing#App_Usage_Ping
[https://wiki.mozilla.org/B2G/QA/Ping_Testing#App_Usage_Ping App Usage]


|-  
|-  
| Moztrap ||  
| Moztrap ||  
https://moztrap.mozilla.org/manage/cases/?filter-suite=775  
[https://moztrap.mozilla.org/manage/cases/?filter-suite=775 Metrics test cases]
|-
|-
| Note ||  
| Note ||  
|}
|}
* SD Card formatting
* MTP


== OTA/FOTA ==
== OTA/FOTA ==
Line 432: Line 508:


== MISC ==
== MISC ==
{| class="wikitable"
|-
| Feature || Market Customization
|-
| QA || [mailto:echang@mozilla.com Eric Chang]
|-
| How to test || You can customize Gaia's default settings, like with a different wallpaper or with support phone already stored inside Contacts, or customize by carrier's MNC, MCC, so the settings can be different if you insert a specific SIM card. See [[B2G/QA/Customizations|wiki]] and [https://developer.mozilla.org/en-US/Firefox_OS/Developing_Gaia/Market_customizations_guide mdn] for more details.
|-
| Moztrap || [https://moztrap.mozilla.org/manage/suites/?pagenumber=1&pagesize=20&sortfield=created_on&sortdirection=desc&filter-name=customized Customized UI Suite] 
|-
| Note || Supported since Firefox OS v1.X
|}

Latest revision as of 09:03, 18 March 2016

Communications

Feature Contacts
QA Isabel Rios
How to test TBD
Moztrap Contacts test case
Note Supported since Firefox OS v1.X
Feature Dialer
QA Isabel Rios
How to test TBD
Moztrap Dialer test case
Note Supported since Firefox OS v1.X
Feature Messages
QA Isabel Rios
How to test You might need to check if your carrier supports MMS sending before testing, some MVNOs don't support MMS.
Moztrap

[1] [2] [3]

Note NowSMS can be used to send SMS, OMA CP... etc
Feature Usage (Cost Control)
QA Isabel Rios
How to test TBD
Moztrap Cost Control
Note Supported since Firefox OS v1.X

Media

Feature
  • Music
  • Gallery
  • Video
  • Camera
  • FM Radio
QA No-Jun Park
How to test
Moztrap
Note Supported since Firefox OS v1.X
Feature Ringtones
QA TBD
How to test Please refer to MozTrap
Moztrap Ringtones test suite
Note Supported since Firefox OS v1.X
Feature Device Storage
QA William Hsu
How to test Please refer to MozTrap
Moztrap Device Storage
Note Supported since Firefox OS v1.X

Multimedia platform

  • Video/Audio:Recording

Productivity

Feature Email
QA Teri Wen
How to test

e-Mail test specification

Moztrap e-Mail RAT test cases
Note Supported since Firefox OS v1.X
Feature Calendar
QA Teri Wen
How to test

Calendar test specification

Moztrap Calendar RAT test cases
Note Supported since Firefox OS v1.X
Feature Clock
QA Teri Wen
How to test

Clock test specification

Moztrap Clock RAT test cases
Note Supported since Firefox OS v1.X

System platform

Feature Lock Screen
QA Gerry Chang
How to test SysPF-Lockscreen test specification
Moztrap

Lock Screen and Phone Lock

Take action on lock screen notifications

Note
Feature Geolocation
QA NA
How to test TBD
Moztrap Geolocation test suites
Note
Feature Settings
QA Iris Hsiao
How to test SysPF-Settings test specification
Moztrap Settings test suites
Note
Feature Keyboard, Built-in Keyboard App, System Keyboard Mgmt
QA Iris Hsiao
How to test SysPF-Keyboard test specification
Moztrap Settings test suites
Note
Feature Text Selection
QA Gerry Chang
How to test Guideline
Moztrap Text Selection test cases
Note
  • Window Management
  • WebRTC
  • Wallpaper
  • DOM
  • Loop testing

System frontend

Feature Homescreen, App-grouping
QA Naoki Hirata
How to test Homescreen test specification
Moztrap [Sys FE] Vertical Homescreen
Note
Feature Browser
QA Alison Shiue
How to test Browser test specification
Moztrap Browser test cases
Note
Feature Notifications
QA NA
How to test TBD
Moztrap Notifications test cases
Note
Feature First Time Experience
QA Alison Shiue
How to test FTE test specification
Moztrap FTE test cases
Note
Feature Task Manager
QA Naoki Hirata
How to test
Moztrap [General]Card View/Task Switcher/Task Manager
Note
Feature Download Manager
QA TBD
How to test Please refer to MozTrap
Moztrap Download manager test cases
Note
Feature Find My Device
QA TBD
How to test Please refer to MozTrap
Moztrap Find My Device test cases
Note
Feature Rocketbar, Search App
QA Naoki Hirata
How to test Rocketbar
Moztrap [General]Rocketbar
Note
Feature Market Customization
QA TBC
How to test You can customize Gaia's default settings, like with a different wallpaper or with support phone already stored inside Contacts, or customize by carrier's MNC, MCC, so the settings can be different if you insert a specific SIM card. See wiki and mdn for more details.
Moztrap Customized UI Suite
Note Supported since Firefox OS v1.X
  • App/Datastore Permissions
  • E.me Integration
  • Offline Error Handling
  • Soft home button
  • Panning and Zooming

Telephony

Module RIL
QA Gerry Chang
How to test Follow the guide to turn on RIL flag for adb debug log
Moztrap RIL test cases
Note
Module Cellbroadcast, CMAS
QA TBC
How to test Currently they can be tested using emulator.
Moztrap

Cell broadcast All test cases

Note

Connectivity

Module RTSP
QA William Hsu
How to test Please refer to MozTrap
Moztrap RTSP test cases
Note
Module NFC
QA Alison Shiue
How to test NFC test specification
Moztrap NFC test cases
Note
Module WIFI
QA Gerry Chang
How to test Please refer to MozTrap
Moztrap WIFI test cases
Note
  • WIFI Direct
  • WIFI Display

Devices

Module Bluetooth
QA Teri Wen, Alison Shiue
How to test Bluetooth PTS certification will be run twice for a release, when in FL and in FC, see PTS wiki for details; See RD's wiki for other info, currently Gaia-ui-test only test on simple bluetooth menu Settings, no automation to test on bluetooth profiles. 
 Bluetooth tooth daemon implemented after V2.2
 adb shell ps bluetoothd
 or 
 adb shell getprop |grep bluetoothd
 then you will see "ro.moz.bluetooth.backend=bluetoothd"
 & [init.svc.bluetoothd]: [running] 
Moztrap [RAT] Bluetooth Cases for running acceptance test in a limited time.

[Sys PF]Settings - Bluetooth Settings menu related test cases.
[BT]Bluetooth All test cases

Profiles supported HSP, HFP, A2DP, AVRCP, IOPT, OPP
Note Previous PTS Meta bug 1064739 for reference, bluetooth is supported since Firefox OS vX.X
Feature Metrics
QA Shing Lyu
How to test

FTU Ping

App Usage

Moztrap

Metrics test cases

Note
  • SD Card formatting
  • MTP

OTA/FOTA

Feature FOTA/ OTA
QA Gerry ChangNaoki Hirata
How to test
Moztrap
Document

Device update channel check

OTA flow chart

Graphics

Feature Graphics
QA No-Jun Park
How to test https://wiki.mozilla.org/B2G/QA/Device_Test_Plan/Graphics
Moztrap https://moztrap.mozilla.org/manage/cases/?filter-suite=669
Sanity Test Log https://docs.google.com/spreadsheets/d/1us2Rw-xwKZvuvvTTidy96C5PPt3ivgB58LSDkmV0qnY/edit#gid=2
Document https://developer.mozilla.org/en-US/docs/Mozilla/QA/Marionette/Run_ImageMagick_Tool_with_Python_Marionette

MISC

Retrieved from "https://wiki.allizom.org/index.php?title=Firefox_OS/FeatureTesting&oldid=1122835"