MozillaWiki

QA/Photon Onboarding Tour Overlay: Difference between revisions

Page Discussion

QA/Photon Onboarding Tour Overlay (view source)

Revision as of 18:30, 13 September 2017

345 bytes removed ,  13 September 2017
→‎Environments
 
(31 intermediate revisions by the same user not shown)
Line 1: Line 1:
'''Approvals Required / Received'''
The following individuals are required to/have approved this Test Plan:
{| class="wikitable"
|-
! Name !! Title !! Department !! Approval Date !! Method
|-
|  Lawrence Mandel / RyanVM || QA Reviewer || Product Integrity || || [mailto:lmandel@mozilla.com Lawrence Mandel], [mailto:rvandermeulen@mozilla.com RyanVM]
|-
| TPE Eng Team || Software Engineer || Engineering || || [mailto:gasolin@mozilla.com Fred Lin], [mailto:fliu@mozilla.com Fischer Liu], [mailto:rexboy@mozilla.com Rex Lee]
|-
| Francis Lee || EPM || Product Management || || [mailto:frlee@mozilla.com Francis Lee]
|}
'''Revision History'''
'''Revision History'''


Line 34: Line 18:
Gives a new user/user that has not used Firefox in a while a description and an introduction to Firefox's new innovative features, such as: <br />
Gives a new user/user that has not used Firefox in a while a description and an introduction to Firefox's new innovative features, such as: <br />


'''V1'''
'''V1 New User'''
* Private Browsing
* Private Browsing
* Addons
* Addons
Line 42: Line 26:
* Sync
* Sync


'''V2'''
'''V2 New User'''
* Performance
* Private Browsing
* Screenshots
* Add-ons
* Add-ons
* Customize
* Customize
* Default Browser
* Default Browser
'''V2 Updated User'''
* Performance
* Library
* Library
* Performance
* Private Browsing
* Screenshots
* Screenshots
* Search
* Search
* Customize
* Sync
* Sync


with a overlay that is accessed by clicking the fox head on the about:newtab page or the default about:home page. The overlay is also accessed by clicking the Onboarding Tour Notification.
with a overlay that is accessed by clicking the Firefox icon or the watermark on the about:newtab page or the default about:home page. The overlay is also accessed by clicking the Onboarding Tour Notification.
 
== Scope ==
Photon Onboarding Tour Overlay will solely focus on the flow of the overlay.


== Ownership ==
== Ownership ==


  Engineering lead
  Engineering lead
   [mailto:jdolske@mozilla.com Justin Dolske]
   [mailto:tchien@mozilla.com Tim Chien]


  Engineering Team
  Engineering Team
Line 78: Line 64:
== Scope of Testing ==
== Scope of Testing ==
=== In Scope ===
=== In Scope ===
Onboarding Tour Overlay which includes:
1. Profile Refresh<br />
2. Onboarding Tour Overlay which includes:
 
'''V1 New User'''
* Private Browsing
* Private Browsing
* Addons
* Addons
Line 85: Line 74:
* Default Browser
* Default Browser
* Sync
* Sync
'''V2 New User'''
* Performance
* Private Browsing
* Screenshots
* Add-ons
* Customize
* Default Browser
'''V2 Updated User'''
* Performance
* Library
* Screenshots
* Search
* Customize
* Sync
These features above are shown in the overlay to give the user a walk through/introduction of Firefox’s innovative features.


=== Out of Scope ===
=== Out of Scope ===
Everything not coupled with the Photon Onboarding Tour Overlay is out of scope.
Everything that does not include the features in scope, about:newtab, about:home, profile refresh, and the hamburger menu. Fennec is also out of scope.
Fennec is also out of scope.


= Requirements for testing =
= Requirements for testing =
== Environments ==
== Environments ==
  Windows 7 & 10
  Windows 7 x64: Intel(R) Core(TM) i5-7500 CPU @ 3.40GHz
  Ubuntu 17.04
Windows 10 x64: Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz, 3601 Mhz
  Mac 10.12
  Ubuntu 17.04 x64:Intel(R) Core(TM) i5-7500 CPU @ 3.40GHz
  Mac 10.12 x64: 2.7 GHz Intel Core i5


= Test Strategy =  
= Test Strategy =  
Line 103: Line 110:
! ID !! Description / Threat Description !! Covered by Test Objective !!  Magnitude !! Probability !! Priority !! Impact Score  
! ID !! Description / Threat Description !! Covered by Test Objective !!  Magnitude !! Probability !! Priority !! Impact Score  
|-
|-
| RAC-1 || User is irritated by not being able to quickly dismiss the onboarding overlay || TO-1 || 2-Moderate || 1-Unlikely || 1-Low || 2
| RAC-1 || User is irritated by not being able to quickly dismiss the onboarding overlay || TO-1 || 2-Moderate || 1-Unlikely || 1-Low || 2
|-
| RAC-2 || Tour shows when updating from 55 to 56 || TO-2 || 3-High || 1-Unlikely || 3-High || 9
|-
| RAC-3 || New user tour does not display with a new profile when 56 and 57 are launched || TO-2 || 3-High || 1-Unlikely || 3-High || 9
|-
|-
| RAC-2 || Tour overlay shows when updating from 55 to 56 (Tour will show for update from 56 to 57) || TO-2 || 3-High || 2-Possible || 3-High || 18
| RAC-4 || Updated User tour does not display when user updates from 56 to 57 || TO-2 || 3-High || 1-Unlikely || 3-High || 9
|-
|-
| RAC-3 || User sees the tour overlay once when updating from 55 to 56 [https://bugzilla.mozilla.org/show_bug.cgi?id=1377470 Bug 1377470] || TO-2 || 3-High || 2-Possible || 3-High || 18
| RAC-5 || Currently no automation test coverage || || 3-High || 2-Possible || 2-Medium || 12
|}
|}


Line 131: Line 142:
! Ref !! Function !! Test Objective !! Evaluation Criteria !! Test Type !! RAC !! Owners  
! Ref !! Function !! Test Objective !! Evaluation Criteria !! Test Type !! RAC !! Owners  
|-
|-
| 1 || Click outside of tour overlay || Tour overlay closes when user clicks out of tour overlay || The tour overlay successfully closes when the user clicks out of the tour overlay || Manual || RAC-1 || Eng Team
| 1 || Click outside of tour overlay || Tour overlay is easily dismissed/closed || The tour overlay successfully closes when the user clicks out of the tour overlay or the x in the top right corner || Manual || RAC-1 || Eng Team
|-
|-
| 2 || Updating Firefox || Updates to Firefox show the tour overlay only when user updates from a version less than 55 to 56. (Tour will show when updating from 56 to 57) || The tour overlay is not shown when the user updates from 55 to 56. The tour overlay is shown when the user updates from 54 to 56 or 56 to 57. || Manual || RAC-2, RAC-3 || Eng Team
| 2 || Updating Firefox || Updates to Firefox show the tour overlay only when user updates from a version less than 55 to 56. (Tour will show when updating from 56 to 57) || The tour overlay is not shown when the user updates from 55 to 56. The tour overlay is shown when the user updates from 54 to 56 or 56 to 57. || Manual || RAC-2, RAC-3 || Eng Team
Line 187: Line 198:
| Test case creation V2 || [https://testrail.stage.mozaws.net/index.php?/suites/view/1372&group_by=cases:section_id&group_order=asc Onboarding Test Case Creation V2]
| Test case creation V2 || [https://testrail.stage.mozaws.net/index.php?/suites/view/1372&group_by=cases:section_id&group_order=asc Onboarding Test Case Creation V2]
|-
|-
| Test case execution V2 || [https://testrail.stage.mozaws.net/index.php?/plans/view/5624 Onboarding Test Case Execution V2]
| Test case execution V2 || [https://testrail.stage.mozaws.net/index.php?/plans/view/5708 Onboarding Test Case Execution V2]
|-
|-
| Bugs management || [https://bugzilla.mozilla.org/show_bug.cgi?id=1354046 Overlay Meta Bug]
| Bugs management || [https://bugzilla.mozilla.org/show_bug.cgi?id=1354046 Overlay Meta Bug]
|-
| Trello || [https://trello.com/c/oMvvrfVD/284-photon-onboarding-overlay-includes-6-original-tours Overlay Trello Card]
|-
| Specs V1 || [https://mozilla.invisionapp.com/share/4MBDUMS5W#/screens/228511972_Overview Overlay V1 Specs]
|-
| Specs V2 || [https://www.google.com/url?q=https://docs.google.com/document/d/1XVowXtnAzzzyLcwdFiP6cx3l9_CNBadP9_7v17XJ40s/edit&sa=D&ust=1504820227394000&usg=AFQjCNEAJ_n8hFKCm_xt-3vtyrtrZETWGg Overlay V2 Specs]
|}
|}


Line 209: Line 226:
|-
|-
| Nightly V2  
| Nightly V2  
|style="text-align:center;"|   ||  
|style="text-align:center;"| 08/18/2017  || 20170818100226
|-
|-
| Beta V2  
| Beta V2  
Line 217: Line 234:
|style="text-align:center;"|  ||  
|style="text-align:center;"|  ||  
|}
|}
= References =
* [https://trello.com/c/oMvvrfVD/284-photon-onboarding-overlay-includes-6-original-tours Overlay Trello Card]
* [https://mozilla.invisionapp.com/share/4MBDUMS5W#/screens/228511972_Overview Overlay Specs]


= Testcases =  
= Testcases =  
Line 256: Line 269:
|-
|-
| Usability and/or discoverability testing   
| Usability and/or discoverability testing   
|style="text-align:center;" | Y || ||
|style="text-align:center;" | Y || Verify that the product is user friendly || TPE Eng Team
|-
|-
| RTL build testing   
| RTL build testing   
Line 294: Line 307:
|style="text-align:center;" | Y || Private Browsing, Sync || TPE Eng Team
|style="text-align:center;" | Y || Private Browsing, Sync || TPE Eng Team
|}
|}
== Test suite ==
Full Test suite - [https://testrail.stage.mozaws.net/index.php?/suites/view/1204&group_by=cases:section_id&group_order=asc Onboarding Test Suite V1]
Full Test suite - [https://testrail.stage.mozaws.net/index.php?/suites/view/1372&group_by=cases:section_id&group_order=asc Onboarding Test Suite V2]
= Bug Work =
Meta bug: [https://bugzilla.mozilla.org/show_bug.cgi?id=1354046 1354046 - bug summary]
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
====== Logged bugs ( blocking [https://bugzilla.mozilla.org/show_bug.cgi?id=1354046 1354046] )======
<div class="mw-collapsible-content">
<bugzilla>
    {
        "blocks":[1354046],
        "include_fields": "id, priority, component, assigned_to, summary, status, target_milestone"
    }
</bugzilla>
</div>
</div>
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
====== Bug fix verification ======
<div class="mw-collapsible-content">
<bugzilla>
    {
        "blocks":[1354046],
        "resolution":"FIXED",
        "include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"
    }
</bugzilla>
</div>
</div>


= Sign off =
= Sign off =
Line 358: Line 337:
Win 7 & 10, Ubuntu 16.04, and Mac 10.12
Win 7 & 10, Ubuntu 16.04, and Mac 10.12
*Link for the tests run
*Link for the tests run
** Full Test suite, link to TestRail - Tests Runs and Results [https://testrail.stage.mozaws.net/index.php?/plans/view/5624 Onboarding Test Run V2]
** Full Test suite, link to TestRail - Tests Runs and Results [https://testrail.stage.mozaws.net/index.php?/plans/view/5708 Onboarding Test Run V2]


<br />
<br />
Line 439: Line 418:
|-
|-
|  QA mid-Nightly Signoff V2
|  QA mid-Nightly Signoff V2
|style="text-align:center;"  |  {{in progress|}}  
|style="text-align:center;"  |  {{mdone|}}  
|style="text-align:center;"| Email to be sent
|style="text-align:center;"| Email sent
|-
|-
|  QA Nightly - Full Testing V2
|  QA Nightly - Full Testing V2
Line 456: Line 435:
|  QA pre-Release Signoff V2 ||  
|  QA pre-Release Signoff V2 ||  
|style="text-align:center;" | Email to be sent  
|style="text-align:center;" | Email to be sent  
|}
'''Approvals Required / Received'''
The following individuals are required to/have approved this Test Plan:
{| class="wikitable"
|-
! Name !! Title !! Department !! Approval Date !! Method
|-
|  Lawrence Mandel / RyanVM || QA Reviewer || Product Integrity || || Email
|-
| Tim Chien || Software Engineer || Engineering || 09/06/2017 || Email
|-
| Cindy Hsiang || PM || Product Management || 09/07/2017 || Email
|}
|}
Jwilliams7
355

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/1178773...1180401"