B2G/QA/Gaia Test Plan/Browser: Difference between revisions

Jump to navigation Jump to search

B2G/QA/Gaia Test Plan/Browser (view source)

Revision as of 00:32, 10 April 2013

7,922 bytes added ,  10 April 2013
no edit summary
(Created page with "== General Information == {| class="fullwidth-table" |- ! style="width: 40%;" | '''Area''' ! style="width: 60%;" | '''Details''' |- | '''Team:''' | *'''Developers:''' *'''D...")
 
No edit summary
Line 14: Line 14:


= Test Plan =
= Test Plan =
== Design Documentation/Background Research ==
== Introduction ==
This test plan is a test plan for testing the browser app in the Gaia portion of b2g.  This documentation will describe the testing strategy that will be used to validate the quality of the browser app.  Included are the various resources required to complete the testing.


== System Testing ==
=== Design Documentation/Background Research ===
== Mobile OS ==
Listed are the related documentation for testing:<br>
* B2G - Firefox OS 18
==== Module Owners ====
* [https://wiki.mozilla.org/Modules Modules]
* [https://docs.google.com/spreadsheet/ccc?key=0AiBigu584YY7dGlNSlY0QzhJb3M5anRBa1gxalV0Y3c#gid=13 B2G - Milestone 5 Plan]
==== PM 1.0.1 Requirements: ====
* https://docs.google.com/spreadsheet/ccc?key=0Ajt_meMlK3MOdHphdnV2a2RvcU9MZmd3cDlIQnZBMVE#gid=0
==== PM 1.1.0 Requirements: ====
* https://docs.google.com/spreadsheet/ccc?key=0AtVT90hlMtdSdGw5QzNLZDczeWRFZ2JIOWJMUUZzSWc#gid=0
==== Bugs tracking features: ====
* For leo : https://bugzilla.mozilla.org/buglist.cgi?keywords=feature&keywords_type=allwords&list_id=6183586&query_based_on=features%2C%20leo%2B&chfieldto=Now&chfield=%5bBug%20creation%5d&query_format=advanced&component=Gaia%3A%3Abrowser&product=Boot2Gecko&known_name=features%2C%20leo%2B
 
==== Specs: ====
* [https://www.dropbox.com/sh/retlarkzknj7luc/QOq2Rv4aXt Visuals]
* [https://www.dropbox.com/sh/1moictrrtb6rwvv/eDKIDJpuqO/Apps/Browser Interactions]
* [http://buildingfirefoxos.com/ Building Blocks for any app in Firefox OS]
** Wiki in Gaia System Specs is outdated
* [https://developer.mozilla.org/en-US/docs/WebAPI/Device_Storage_API Device Storage API]
* [https://wiki.mozilla.org/Gaia Gaia System Specs]
 
==== QA Mapping ====
* [https://docs.google.com/spreadsheet/ccc?key=0AqDJxnTd7VDLdG05cXR1Y01zc2pMeVlHaHlhOElRN3c#gid=0 Testcase Writing Spreadsheet]
 
==== Module Owners ====
* [https://wiki.mozilla.org/Modules Modules]
* [https://docs.google.com/spreadsheet/ccc?key=0AiBigu584YY7dGlNSlY0QzhJb3M5anRBa1gxalV0Y3c#gid=13 B2G - Milestone 5 Plan]
 
==== Automated Testing: ====
* [https://air.mozilla.org/marionette/ Video on Marionette introduction]
* [https://air.mozilla.org/b2gfirefoxos-front-end-automation-in-python/ Video on Front end automation on b2g using marionette]
* [https://github.com/mozilla/gaia-ui-tests Repository for Front end tests]
* [https://developer.mozilla.org/en-US/docs/Marionette/Marionette documentation on Marionette API]
* [https://developer.mozilla.org/en-US/docs/Marionette/Marionette_Python_Tests Documentation on how to write a python test for Marionette]
* [https://wiki.mozilla.org/Auto-tools/Projects/Marionette/JSON_Protocol JSON Protocol for Marionette]
 
==== Repositories: ====
* [https://hg.mozilla.org/releases/mozilla-b2g18/ mercurial repo of gecko]
* [https://hg.mozilla.org/integration/gaia/ mercurial repo of gaia]
* [https://github.com/mozilla-b2g/gaia/tree/v1-train github repo of v1 train]
* [https://hg.mozilla.org/gaia-l10n mercurial repo of l10n translations]
* [https://github.com/mozilla-b2g/gaia/tree/master/locales github repo of locales and how to build multilocale Gaia]
 
==== Tips and Tricks for testing: ====
* https://wiki.mozilla.org/B2G/QA/Tips_And_Tricks
* https://developer.mozilla.org/en-US/docs/Performance/Profiling_with_the_Built-in_Profiler
 
==== OS Diagrams ====
* https://github.com/sotaroikeda/firefox-diagrams/wiki/Firefox-Diagrams
 
==== Meetings ====
* [https://mozqa.etherpad.mozilla.org/b2g-qa-roundtable qa round table meetings schedule]
* [https://mozqa.etherpad.mozilla.org/b2g-standup b2g qa weekly standups]
* [https://etherpad.mozilla.org/gaia-meeting-notes gaia crossfunctional weekly standups]
* [https://etherpad.mozilla.org/b2g-meeting-notes b2g crossfunctional team weekly standups]
 
=== Schedule Milestones ===
Scheduling is based on the milestone listed on the Release Management wiki for B2G Landings:
*[https://wiki.mozilla.org/Release_Management/B2G_Landing https://wiki.mozilla.org/Release_Management/B2G_Landing]
 
Features that are required to land are listed here :
* [https://bugzilla.mozilla.org/buglist.cgi?f1=cf_blocking_b2g&list_id=6146936&short_desc=User%20Story&o1=substring&classification=Client%20Software&query_format=advanced&short_desc_type=allwordssubstr&v1=leo%2B&component=Gaia%3A%3AE-Mail&product=Boot2Gecko Leo Milestone]
 
Lower priority bugs that may or may not make it into the milestones:
* [https://bugzilla.mozilla.org/buglist.cgi?f1=cf_blocking_b2g&list_id=6146960&short_desc=User%20Story&o1=notequals&classification=Client%20Software&query_format=advanced&short_desc_type=allwordssubstr&v1=leo%2B&component=Gaia%3A%3AE-Mail&product=Boot2Gecko Leo milestone]
 
== Hardware Requirements ==
* A desktop machine capable of running ADB is necessary for testing for getting debug information and flash the device.
* Mobile devices listed
* Optional: Build machine to build your own [https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/Firefox_OS_build_prerequisites?redirectlocale=en-US&redirectslug=Firefox_OS%2FFirefox_OS_build_prerequisites gaia/gecko]
* Optional: Build OS Storage machine to make it easier to find regressions
=== Mobile Device ===
=== Mobile Device ===
Test devices :  
Test devices :  
* Otoro
* Otoro
* Unagi
* Unagi
== Software Requirements ==
* [http://developer.android.com/sdk/index.html ADB]
* [https://pvtbuilds.mozilla.org/pub/mozilla.org/b2g/nightly/ B2G OS Builds]
* Python 2.6 for Marionette
== System Testing ==
=== Mobile OS  ===
* B2G - Firefox OS 18
=== Accounts ===
* See [https://intranet.mozilla.org/QA#Test_Accounts Test Accounts]
* Facebook Contact Import accounts also listed in the above link


== Strategy ==
== Strategy ==
* Exploratory Manual testing using outline; see Test Outline section
* Smoke Tests are run daily for sanity check on builds
* Regression testing based on Test Cases
* Exploratory Manual  
* Eventual automation testing
** Modified [http://en.wikipedia.org/wiki/Session-based_testing Session Based] testing around:
*** testing using outline; see Test Outline section
*** Bug fixes
*** Features implemented
* Regression Testing based on Test Cases
** Found via bugs
** based on outline/feature/spec
** Test cases will be tracked in MozTrap under "gaia" and "browser"
* Automation testing:
** [https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/Platform/Testing/Gaia_unit_tests Unit testing] ( this is handled by developers)
** [https://blog.mozilla.org/webdev/2012/10/19/pre-integration-testing/ Other Preintegration testing] such as [https://travis-ci.org/mozilla-b2g/gaia Travis Continual Integration Testing] (this is handled by developers)
** Marionette testing as covered in related documents
* Performance Testing
** [https://datazilla.mozilla.org/b2g/?branch=master&range=7&test=cold_load_time&app_list=email,music,phone,settings,usage,video&app=phone&gaia_rev=10627f87346ffce9&gecko_rev=9772aa4180581d7e Perfomatic]
** Note: we do need different specific email performance testing and there is a gap currently


== Signoff Criteria ==
* [https://wiki.mozilla.org/B2G/Triage Triage of bugs]
* All basecamp blockers are closed
** [https://wiki.mozilla.org/B2G/QA/Triage QA Response to Triaged bugs]
* All smoke tests and basic functional tests are ran without finding any basecamp blockers


== Test Cases and Results ==
Currently not tested:
* Test cases will be tracked in MozTrap under "gaia" and "email"
* Performance against different mobile data types
 
== Test Deliverables ==
* Test Cases for the User Stories before end of feature complete
* Test Runs for each phase of the project
* Smoke test runs on a daily basis
* Automation Tests for future regression testing
 
== Dependencies/Risk ==
* Some Marionette features need to land before certain automation tests can be written.  (ie swiping)
* Test Runs w/ tests on the areas of change can only commence once the features have been complete.
 
== Signoff/Exit Criteria ==
Feature Complete :
* All User Story with leo+ blocking are closed and verified
 
Release:
* All smoke tests and basic functional tests are ran without finding any blockers
* All leo+ blockers that are fixed are verified


= Test Outline =
= Test Outline =
Line 43: Line 155:
=== UI ===
=== UI ===
* General  
* General  
 
** Browser Settings
* Gestures
*** About Browser
 
*** Clear browsing history
* Keyboard Short Cuts
*** Clear Cookies and stored data
** Tab Tray
*** add tab
** Awesome Page
*** URL bar
**** Go
**** return
**** Cancel
**** type ahead
**** search
*** Top Sites
*** Bookmark
*** History
*** unbookmark (long tap gesture)
** Bookmark
*** Selected
**** Bookmark/Unbookmark
**** Add to Home Screen
**** Cancel
**** Edit Bookmark


=== Functional ===
=== Functional ===
Line 52: Line 183:
* Input
* Input
** Symbols:
** Symbols:
*** Astricks () {} [] & ^ % $ # @ ! ~ etc.
*** Astricks () {} [] & ^ % $ # @ ! ~ etc.
**** Note: in email address and passwords esp.
** Screen size (making sure that things fit)
** Screen size (making sure that things fit)
* folder names in foreign characters


* Conditional Functionality
* Conditional Functionality
** Browser Settings
*** About Browser
*** Clear browsing history
*** Clear Cookies and stored data
** Tab Tray
*** add tab
** Awesome Page
*** URL bar
**** Go
**** return
**** Cancel
**** type ahead
**** search
*** Top Sites
*** Bookmark
*** History
*** unbookmark (long tap gesture)
** Bookmark
*** Selected
**** Bookmark/Unbookmark
**** Add to Home Screen
**** Cancel
**** Edit Bookmark
* Content
** long tap on video
** long tap on music


== Beta Conditional  ==
== Beta Conditional  ==
Line 61: Line 220:


*Content Conditions
*Content Conditions
** Phishing issues?


*Physical Conditions  
*Physical Conditions  
Line 79: Line 239:
** Security
** Security
*** Buffer Overflow (max characters)
*** Buffer Overflow (max characters)
** removal of characters


*Race Conditions  
*Race Conditions  
Line 156: Line 315:


* Help?
* Help?
* Performance


== Usability ==
== Usability ==
*Usability
*Usability
* Accessibility (508 compliance)
* Accessibility (508 compliance)
Nhirata
Confirmed users
4,378

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/645355"

Navigation menu