MozillaWiki

Firefox OS/DeviceQA/DeviceQAWiki: Difference between revisions

Page Discussion

Firefox OS/DeviceQA/DeviceQAWiki (view source)

Revision as of 13:49, 1 February 2015

1,275 bytes added ,  1 February 2015
m
Lakrits moved page FirefoxOS/DeviceQA/DeviceQAWiki to Firefox OS/DeviceQA/DeviceQAWiki: The official spelling of "Firefox OS" leaves a space between the two parts of the name. It's easier to find a page if the spelling of its name is consistent...
m (Lakrits moved page FirefoxOS/DeviceQA/DeviceQAWiki to Firefox OS/DeviceQA/DeviceQAWiki: The official spelling of "Firefox OS" leaves a space between the two parts of the name. It's easier to find a page if the spelling of its name is consistent...)
 
(10 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== Device QA Wiki ==
== Note of handling device project ==
 
=== Note of handling device project ===
* Weekly report should be delivered to stakeholders.  
* Weekly report should be delivered to stakeholders.  
* "[https://mail.mozilla.org/listinfo/fxosqa-report-watchlist fxosqa-report-watchlist]" is a public mailing list, please consider if report should be delivered to them.
* "[https://mail.mozilla.org/listinfo/fxosqa-report-watchlist fxosqa-report-watchlist]" is a public mailing list, please consider if report should be delivered to them.


=== Manual Test ===
== Manual Test ==
* Sanity Test
* Sanity Test
* Smoke Test (Manual/Auto)
* Smoke Test (Manual/Auto)
* Acceptance Test
* [https://moztrap.mozilla.org/manage/cases/?&pagenumber=1&pagesize=100&sortfield=created_on&sortdirection=desc&filter-status=active&filter-tag=2862 Acceptance Test]
* CRT (Certification Requirement Test) [TBD]
* CRT (Certification Requirement Test) [TBD]
* Functional Full Test
* Functional Full Test
Line 16: Line 14:
* RAT (Release Acceptance Test)
* RAT (Release Acceptance Test)


=== Automation ===
== Automation ==
When a project kick-off, Device QA team will start to arrange following environment for testing. Partners should aware it's not required. However, we still recommend partners setup the same environment for quick responding issues found at both sides. We also suggest to discuss about which testing you'd like to have and their pass criteria with TAM before a project kick-off.
When a project kick-off, Device QA team will start to arrange following environment for testing. Partners should aware it's not required. However, we still recommend partners setup the same environment for quick responding issues found at both sides. We also suggest to discuss about which testing you'd like to have and their pass criteria with TAM before a project kick-off.
==== Functionality ====
=== Functionality ===
* Gaia-UI-Test
* Gaia-UI-Test
** gaia-ui-tests cover functionality testing by checking and interacting with HTML elements rendered on screen. Device QA team suggests to execute on real devices.
** gaia-ui-tests cover functionality testing by checking and interacting with HTML elements rendered on screen. Device QA team suggests to execute on real devices.
** Suggested Pass Criteria: All green for five days
** Suggested Pass Criteria: All green for five days
** [https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/gaia-ui-tests MDN: Gaia-ui-test]
** [https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/gaia-ui-tests MDN: Gaia-ui-test]
==== Stability ====
=== Stability ===
* MTBF (Mean Time Between Failures)
* MTBF (Mean Time Between Failures)
** MTBF is often used for measuring stability. Failures mentioned here include crashes and hang.
** MTBF is often used for measuring stability. Failures mentioned here include crashes and hang.
** Suggested Pass Criteria: MTBF > 100 hours
** Suggested Pass Criteria: MTBF > 100 hours
** [https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/MTBF_tests MDN: MTBF]
** [https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/MTBF_tests MDN: MTBF]
** [https://wiki.mozilla.org/QA/Automation/Projects/MTBF Wiki: MTBF]
* Monkey Test
* Monkey Test
** Monkey Test of Firefox OS is based on [https://github.com/mozilla-b2g/orangutan Orangutan tool].
** Monkey Test of Firefox OS is based on [https://github.com/mozilla-b2g/orangutan Orangutan tool].
Line 35: Line 34:
** Endurance won't be part of our regular tests since it takes more time. You can have a look of endurance testing result of our reference phone (Flame) from [https://datazilla.mozilla.org/ datazilla].
** Endurance won't be part of our regular tests since it takes more time. You can have a look of endurance testing result of our reference phone (Flame) from [https://datazilla.mozilla.org/ datazilla].
** [https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/Endurance MDN: Endurance]
** [https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/Endurance MDN: Endurance]
==== Performance ====
 
=== Performance ===
* Performance (not maintained in Taipei)
* Performance (not maintained in Taipei)
** You can have a look of performance result of our reference phone (Flame) from [https://datazilla.mozilla.org/ datazilla]
** You can have a look of performance result of our reference phone (Flame) from [https://datazilla.mozilla.org/ datazilla]
** [https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/Gaia_performance_tests MDN: Gaia UI Performance]
** [https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/Gaia_performance_tests MDN: Gaia UI Performance]
==== Certification ====
=== Certification ===
* MCTS (Mozilla Certificate Test Suite)
* MCTS (Mozilla Certificate Test Suite)
== UX Reference ==
=== Firefox OS UX ===
. v2.1: https://mozilla.app.box.com/s/44utizl9oz4eupyu3fuu/1/1759289051
. v2.0: https://mozilla.app.box.com/s/44utizl9oz4eupyu3fuu/1/1732424767
. v1.4: https://mozilla.app.box.com/s/44utizl9oz4eupyu3fuu/1/1226137087
. v1.3: https://mozilla.app.box.com/s/44utizl9oz4eupyu3fuu/1/1147153091
. v1.2: https://mozilla.app.box.com/s/44utizl9oz4eupyu3fuu/1/1030015572
=== Productivity ===
Email (Juwei) : https://mozilla.box.com/s/ltg9nwbw134p8e0hzrnr
Calendar (Harly) : https://mozilla.box.com/s/as74q673zgr7c7d28xeh
Clock (Juwei) : https://mozilla.box.com/s/x1b7dsp7v21sorvp65gr
=== Comms ===
Dialer (Carrie) : https://mozilla.app.box.com/files/0/f/1803746785/Dialer
Message (Jenny) : https://mozilla.app.box.com/files/0/f/1966102506/Messages
Contact (Carrie) : https://mozilla.app.box.com/files/0/f/2107691536/Contact
=== System ===
Keyboard & Text Selection (Omega) : https://mozilla.box.com/s/6cu12ppxdr2m0czuurel
Settings (Jenny) : https://mozilla.box.com/s/og17l1iy5so2akk5xeo0
NFC (Juwei) : https://mozilla.box.com/s/u1gxr4io329zn3m27kq1
Lakrits
3,860

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/1013728...1052427"