MozillaWiki

QA/Photon Preferences Search: Difference between revisions

Page Discussion

QA/Photon Preferences Search (view source)

Revision as of 08:41, 14 September 2017

610 bytes added ,  14 September 2017
no edit summary
No edit summary
 
(25 intermediate revisions by 2 users not shown)
Line 1: Line 1:
'''Approvals Required / Received'''
The following individuals are required to/have approved this Test Plan:
{| class="wikitable"
|-
! Name !! Title !! Department !! Approval Date !! Method
|-
| Lawrence Mandel || QA Manager || Product Integrity ||  ||
|-
| Evan Tseng|| Software Engineer || Engineering ||  || [mailto:evanxd@mozilla.com Email]
|-
| Francis Lee || EPM || Product Management ||  || [mailto:frlee@mozilla.com Email]
|}
'''Revision History'''
'''Revision History'''


Line 23: Line 38:


== Scope ==
== Scope ==
This wiki details the testing that will be performed by the project team for the New DevTools Debugger Panel project. It defines the overall testing requirements and provides an integrated view of the project test activities.  Its purpose is to document what and how we shall test:
This wiki details the testing that will be performed by the project team for the Photon Preferences Search. It defines the overall testing requirements and provides an integrated view of the project test activities.  Its purpose is to document what and how we shall test:
* What will be tested
* What will be tested
* How testing will be performed
* How testing will be performed
Line 31: Line 46:
'''<big>Development Team</big>''':<br />  
'''<big>Development Team</big>''':<br />  
Product Manager: [mailto:frlee@mozilla.com Francis Lee] <br />
Product Manager: [mailto:frlee@mozilla.com Francis Lee] <br />
Engineer: [mailto:evanxd@mozilla.com Evan Tseng] (irc: evanxd)<br />
Engineer: [mailto:evanxd@mozilla.com Francis Lee] (irc: evanxd)<br />
Engineer: [mailto:rchien@mozilla.com Ricky Chien] (irc: rickychien)<br />
Engineer: [mailto:rchien@mozilla.com Ricky Chien] (irc: rickychien)<br />


Line 39: Line 54:
[mailto:hani.yacoub@softvision.ro Hani Yacoub] (irc: hani) - QA <br />
[mailto:hani.yacoub@softvision.ro Hani Yacoub] (irc: hani) - QA <br />
[mailto:roxana.leitan@softvision.ro Roxana Leitan] (irc: RoxanaLeitan) - QA <br />
[mailto:roxana.leitan@softvision.ro Roxana Leitan] (irc: RoxanaLeitan) - QA <br />
'''<big>Release QA Team</big> - Beta Testing''':<br />
[mailto:camelia.badau@softvision.ro Camelia Badau] (irc: cbadau) - QA <br />
[mailto:bogdan.maris@softvision.ro Bogdan Maris] (irc: bogdan_maris) - QA <br />


= Testing summary =  
= Testing summary =  
Line 58: Line 77:
* No search results
* No search results
* Search highlight and Firefox support positioning
* Search highlight and Firefox support positioning
=== Out of Scope ===
* Hover state of the search field
* The state of the search field after pressing on "Enter"
* The state of the search field after clicking outside of the field
* Live search results and suggestion list


= Requirements for testing =
= Requirements for testing =
Line 63: Line 88:
Full testing will be performed on the following OSes:
Full testing will be performed on the following OSes:
* Windows 10 x64
* Windows 10 x64
* Windows 7 x64
* Mac OS X 10.12
* Mac OS X 10.12
* Ubuntu 16.04
* Ubuntu 16.04
Line 92: Line 118:
! ID !! Description / Threat Description !! Covered by Test Objective !!  Magnitude !! Probability !! Priority !! Impact Score  
! ID !! Description / Threat Description !! Covered by Test Objective !!  Magnitude !! Probability !! Priority !! Impact Score  
|-
|-
| RAC-1 || Search field behavior || TO-1 || 1-High || 2-Possible || 3-High || 12
| RAC-1 || Search field behavior could be different than the desirable one || TO-1 || 3-High || 2-Possible || 3-High || 18
|-
|-
| RAC-2 || Not all search results are highlighted || TO-2 || 2-Moderate || 2-Possible || 3-High || 18
| RAC-2 || Not all search results are highlighted || TO-2 || 2-Moderate || 2-Possible || 3-High || 12
|-
|-
| RAC-3 || An indicator may not correctly point the position of the result when it is hiding in a UI component || TO-3 || 2-Moderate || 2-Possible || 3-High || 18
| RAC-3 || An indicator may not correctly point the position of the result when it is hiding in a UI component || TO-3 || 2-Moderate || 2-Possible || 3-High || 12
|}
|}


Line 130: Line 156:
| 5 || Search highlight and Firefox support positioning || All search results should be highlighted according to the spec and the positioning of page element || Manual || RAC-1, RAC-2 || Eng Team
| 5 || Search highlight and Firefox support positioning || All search results should be highlighted according to the spec and the positioning of page element || Manual || RAC-1, RAC-2 || Eng Team
|-
|-
| 6 || Accessibility ||Testing for accessibility includes:
| 6 || Accessibility || Ensuring that preference search functions accordingly with screen readers, in high contrast mode, and using the keyboard exclusively || Manual || RAC-1, RAC-2 || Eng Team
- testing using high Contract themes <br />
- Screen Reader validation<br />
- Keyboard-only 
|| Manual || RAC-1, RAC-2 || Eng Team
|}
|}


Line 140: Line 162:
This section should contain links for builds with the feature -  
This section should contain links for builds with the feature -  
* Links for Nightly builds - [http://archive.mozilla.org/pub/firefox/nightly/2017/04/2017-04-25-03-02-21-mozilla-central// link]
* Links for Nightly builds - [http://archive.mozilla.org/pub/firefox/nightly/2017/04/2017-04-25-03-02-21-mozilla-central// link]
* Links for Beta builds
* Links for Beta builds - [https://archive.mozilla.org/pub/firefox/candidates/56.0b9-candidates/build1/ link]


== Test Execution Schedule ==
== Test Execution Schedule ==
Line 149: Line 171:
|-
|-
| Start project  
| Start project  
|style="text-align:center;" | ||  
|style="text-align:center;" | 04/25/2017 ||  
|-
|-
| Study documentation/specs received from developers
| Study documentation/specs received from developers
|style="text-align:center;" | ||  
|style="text-align:center;" | 04/25/2017 || 04/27/2017
|-
|-
| QA - Test plan creation  
| QA - Test plan creation  
|style="text-align:center;" | ||  
|style="text-align:center;" | 04/25/2017 || 07/25/2017
|-
|-
| QA - Test cases/Env preparation  
| QA - Test cases/Env preparation  
|style="text-align:center;" | ||  
|style="text-align:center;" | 04/27/2017 || 06/20/2017
|-
|-
| QA - Nightly Testing  
| QA - Nightly Testing  
|style="text-align:center;" | ||  
|style="text-align:center;" | 05/08/2017 || 07/28/2017
|-
|-
| QA - Beta Testing  
| QA - Beta Testing  
|style="text-align:center;" | ||  
|style="text-align:center;" | 08/10/2017 || 09/08/2017
|-
|-
| Release Date  
| Release Date  
Line 178: Line 200:
| Test plan creation || Mozilla wiki
| Test plan creation || Mozilla wiki
|-
|-
| Test case creation || [https://testrail.stage.mozaws.net/index.php TestRail]/ Google docs
| Test case creation || [https://testrail.stage.mozaws.net/index.php?/suites/view/933 Test Suite]
|-
|-
| Test case execution || [https://testrail.stage.mozaws.net/index.php TestRail]
| Test case execution || [https://testrail.stage.mozaws.net/index.php?/plans/view/3884 Mid-Nightly56] [https://testrail.stage.mozaws.net/index.php?/plans/view/3884 Pre-Beta56] [https://testrail.stage.mozaws.net/index.php?/plans/view/3884 Pre-Release56]
|-
|-
| Bugs management || Bugzilla
| Bugs management || Bugzilla
Line 187: Line 209:
= Status =  
= Status =  
== Overview ==
== Overview ==
  Track the dates and build number where feature was released to Nightly
  Track the dates and build number where feature was released to Nightly: 2017-06-16
  Track the dates and build number where feature was merged to Release/Beta
  Track the dates and build number where feature was merged to Release/Beta


Line 194: Line 216:
* List and links for specs
* List and links for specs
   - [https://mozilla.invisionapp.com/share/ZDAGPK3AF#/screens/218928219 Preferences Search Design]
   - [https://mozilla.invisionapp.com/share/ZDAGPK3AF#/screens/218928219 Preferences Search Design]
* User Story Used


= Testcases =  
= Testcases =  
Line 329: Line 350:
|-
|-
| Fuzzing   
| Fuzzing   
|style="text-align:center;" | Mo || || Francis Lee  
|style="text-align:center;" | No || || Francis Lee  


|-
|-
Line 356: Line 377:


== Test suite ==
== Test suite ==
  Full Test suite - Link to test rail - tascases should be added under Firefox Desktop project [https://testrail.stage.mozaws.net/index.php?/suites/overview/17 link]
  Full Test suite - Link to test rail - tascases should be added under Firefox Desktop project [https://testrail.stage.mozaws.net/index.php?/suites/view/1217&group_by=cases:section_id&group_order=asc link]
  Smoke Test suite - Link with the tests - if available/needed.
  Smoke Test suite - Link with the tests - if available/needed.
  Regression Test suite - Link with the tests - if available/needed.
  Regression Test suite - Link with the tests - if available/needed.


= Bug Work =
= Bug Work =
Meta bug: [https://bugzilla.mozilla.org/show_bug.cgi?id=1357285 1357285 - bug summary]
Meta bug: [https://bugzilla.mozilla.org/show_bug.cgi?id=1357285 1357285 - [meta] Preferences search]
 
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
====== Logged bugs ( blocking [https://bugzilla.mozilla.org/show_bug.cgi?id=12345 12345] )======
 
<div class="mw-collapsible-content">
<bugzilla>
    {
        "blocks":[12345],
        "include_fields": "id, priority, component, assigned_to, summary, status, target_milestone"
    }
</bugzilla>
 
</div>
</div>
 
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
====== Bug fix verification ======
<div class="mw-collapsible-content">
<bugzilla>
    {
        "blocks":[1357285],
        "resolution":"FIXED",
        "include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"
    }
</bugzilla>
</div>
</div>


= Sign off =
= Sign off =
Line 406: Line 400:
** Regression Test suite, if needed/available
** Regression Test suite, if needed/available


'''Merge to Aurora Sign-off'''
'''Pre-Release Sign-off (2017-09-08)'''<br />
List of OSes that will be covered by testing<br />
 
List of OSes that will be covered by testing
* Windows 7 x64
* Windows 10 x64
* macOS 10.12
* Ubuntu 16.04 x64
 
*Link for the tests run
*Link for the tests run
** Full Test suite
** Full Test suite - Tests Runs and Results [https://testrail.stage.mozaws.net/index.php?/plans/view/3884 link]


== Checklist ==
== Checklist ==
Line 444: Line 444:
|style="text-align:center;" |  
|style="text-align:center;" |  
|-
|-
|  QA Signoff  - Nightly Release||
|  QA Signoff  - Nightly Release  
| |style="text-align:center;" {{mdone|}} || Email to be sent
|style="text-align:center;" |  {{mdone|}} ||  
|-
|  QA Aurora - Full Testing
|style="text-align:center;" |  ||
|-
|  QA Signoff  - Aurora Release ||
|style="text-align:center;"| Email to be sent
|-
|-
|  QA Beta - Full Testing
|  QA Beta - Full Testing
|style="text-align:center;" |  ||  
|style="text-align:center;" |  {{mdone|}} ||  
|-
|-
|  QA Signoff  - Beta Release ||  
|  QA Signoff  - Beta Release ||  
|style="text-align:center;" | Email to be sent
|style="text-align:center;" | {{mdone|}} ||
|}
|}
Camelia.badau
Confirmed users
644

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/1176685...1180435"