MozillaWiki

QA/improve permission prompt UI: Difference between revisions

Page Discussion

QA/improve permission prompt UI (view source)

Revision as of 06:51, 6 April 2017

5,697 bytes added ,  6 April 2017
→‎Checklist
No edit summary
 
(39 intermediate revisions by 2 users not shown)
Line 24: Line 24:


== Scope ==
== Scope ==
This wiki details the testing that will be performed by the project team for the <project name> project. It defines the overall testing requirements and provides an integrated view of the project test activities. Its purpose is to document:
This wiki details the testing that will be performed by the project team for the Permissions Prompt Improvements project. It defines the overall testing requirements and provides an integrated view of the project test activities. Its purpose is to document what and how we shall test:
* What will be tested
* Permissions are listed and correctly managed for devices actively streaming
* How testing will be performed
* The area involving includes every permission doorhanger used in Firefox, with the exception of the plugin prompt.
* Behavior of the permission prompt when the resources (audio/video/sharing) are called from a sub-frame
* Verify permission prompt on secure and unsecured sites.


== Ownership ==
== Ownership ==
'''<big>Development Team</big>''':<br />  
'''<big>Development Team</big>''':<br />  
 
Product Manager: [mailto:pdolanjski@mozilla.com Peter Dolanjski] (irc: pdol)<br />
Project Manager: [mailto:elancaster@mozilla.com Erin Lancaster] (irc: elan)<br />
Engineering Manager: [mailto:pastithas@mozilla.com Panagiotis Astithas] (irc: past)<br />
UX Lead: [mailto:psackl@mozilla.com Philipp Sackl] (irc: phlsa)<br />


'''<big>Engineering Team</big> - Nightly Testing''':<br />  
'''<big>Engineering Team</big> - Nightly Testing''':<br />  
Line 37: Line 42:
[mailto:hani.yacoub@softvision.ro Hani Yacoub] (irc: hani) - QA <br />
[mailto:hani.yacoub@softvision.ro Hani Yacoub] (irc: hani) - QA <br />


= Testing summary =  
= Testing summary =  
== Scope of Testing ==
== Scope of Testing ==
Line 43: Line 47:
This feature purpose is to convert permission notifications to use two buttons instead of the drop-down menu.
This feature purpose is to convert permission notifications to use two buttons instead of the drop-down menu.


Testing will be done on 4 OSes:
The testing effort for the improve permission prompt UI will be invested on the following areas:
* Windows 10, Windows 7
 
* Mac OS X 10.11
* functionality: basic and advanced functionality to be verified according to the existing requirements;
* Ubuntu 16.04
* usability: the Indicator for device permissions must be easy to use and straightforward;
* accessibility: the Indicator for device permissions can be used with High Contrast Theme, Keyboard only or Screen Reader;


In the scope of current testing are following permissions:


=== Out of Scope ===
* Camera
Detail what is out of scope from a testing perspective for the project team. Note: if usability testing is not in the scope of testing feature.
* Microphone
* Screen/Window Sharing
* Desktop Notifications
* Geolocation
* Offilne Storage
* Pop-up Notifications
* Combination between the ones mentioned above


= Requirements for testing =
= Requirements for testing =
== Environments ==
== Environments ==
Full testing will be performed on the following OSes:
Full testing will be performed on the following OSes:
* Windows 7 and Windows 10
* Windows 10 x64, Windows 7 x32
* Mac OS X 10.12
* Mac OS X 10.10, Mac OS X 10.11 and Mac OS X 10.12
* Ubuntu 16.04
* Ubuntu 16.04


== Channel dependent settings (configs) and environment setups ==
== Channel dependent settings (configs) and environment setups ==
Line 65: Line 76:
=== Nightly ===
=== Nightly ===
<div class="mw-collapsible-content">
<div class="mw-collapsible-content">
text
</div>
</div>


=== Aurora ===
=== Aurora ===
<div class="mw-collapsible-content">
<div class="mw-collapsible-content">
text
</div>
</div>


=== Beta ===
=== Beta ===
<div class="mw-collapsible-content">
<div class="mw-collapsible-content">
text
</div>
</div>


=== Post Beta / Release ===
=== Post Beta / Release ===
<div class="mw-collapsible-content">
<div class="mw-collapsible-content">
text
</div>
</div>
</div>
</div>
Line 93: Line 100:
! Ref !! Function !! Test Objective !! Test Type !! Risk Assessment and Coverage(RAC) !! Owners  
! Ref !! Function !! Test Objective !! Test Type !! Risk Assessment and Coverage(RAC) !! Owners  
|-
|-
| 1 || Improve permission prompt UI display || The new visual design is displayed and the notification functionality is correct || Manual || RAC- || Eng Team
| TO-1 || Permission prompt for Camera || The new visual design is displayed and the notification functionality is correct || Manual || RAC-1, RAC-2 || Eng Team
|-
| TO-2 || Permission prompt for Camera functionality || The basic functions(Allowed, Allowed temporarily, Blocked and Blocked temporarily) work properly || Manual || RAC-1, RAC-2 || Eng Team
|-
| TO-3 || Permission prompt for Microphone|| The new visual design is displayed and the notification functionality is correct || Manual || RAC-1, RAC-2 || Eng Team
|-
| TO-4 || Permission prompt for Microphone functionality || The basic functions(Allowed, Allowed temporarily, Blocked and Blocked temporarily) work properly || Manual || RAC-1, RAC-2 || Eng Team
|-
| TO-5 || Permission prompt for Pointer lock|| The new visual design is displayed and the notification functionality is correct || Manual || RAC-1, RAC-2 || Eng Team
|-
| TO-6 || Permission prompt for Pointer lock functionality || The basic functions(Allowed, Allowed temporarily, Blocked and Blocked temporarily) work properly || Manual || RAC-1, RAC-2 || Eng Team
|-
| TO-7 || Permission prompt for Notifications|| The new visual design is displayed and the notification functionality is correct || Manual || RAC-1, RAC-2 || Eng Team
|-
| TO-8 || Permission prompt for Notification functionality|| The basic functions(Allowed, Allowed temporarily, Blocked and Blocked temporarily) work properly || Manual || RAC-1, RAC-2 || Eng Team
|-
| TO-9 || Permission prompt for Location|| The new visual design is displayed and the notification functionality is correct || Manual || RAC-1, RAC-2 || Eng Team
|-
| TO-10 || Permission prompt for Location functionality|| The basic functions(Allowed, Allowed temporarily, Blocked and Blocked temporarily) work properly || Manual || RAC-1, RAC-2 || Eng Team
|-
| TO-11 || Permission prompt for Popups|| The new visual design is displayed and the notification functionality is correct || Manual || RAC-1, RAC-2 || Eng Team
|-
| TO-12 || Permission prompt for Popups functionality|| The basic functions(Allowed, Blocked and Show x blocked pop-up) work properly || Manual || RAC-1, RAC-2 || Eng Team
|-
|-
| 2 || Improve permission prompt UI functionality || The basic functions(Save-don't save, Share-don't share, Allow-don't allow and always remember my decision) all work properly || Manual || RAC- || Eng Team
| TO-13 || Clearing Permissions || Validate that permission prompt set can be cleared correctly || Manual || RAC-1, RAC-2, RAC-3 || Eng Team
|-
|-
| 3 || Different permission prompt types || Validate that new permission prompt design works correctly on different permission prompt types || Manual || RAC- || Eng Team
| TO-14 || Combinations of different permission prompt types || Validate that new permission prompt design works correctly for multiple permission prompt || Manual || RAC-1, RAC-2, RAC-5 || Eng Team
|-
|-
| 4 || Performance is acceptable in case of multiple tabs and doesn't crash the browser || The objective the test is trying to demonstrate that new permission prompt design works correctly when on multiple tabs or windows || Manual || RAC- || Eng Team
| TO-15 || Performance is acceptable in case of multiple tabs and doesn't crash the browser || The objective the test is trying to demonstrate that new permission prompt design works correctly when on multiple tabs or windows || Manual || RAC-4 || Eng Team
|-
|-
| 5 || Accessibility || Testing for accessibility includes:  
| TO-16 || Accessibility || Testing for accessibility includes:  
- testing using high Contract themes <br />
- testing using high Contract themes <br />
- Screen Reader validation<br />
- Screen Reader validation<br />
- Keyboard-only  || Manual || RAC- || Eng Team
- Keyboard-only   
|| Manual || RAC-3, RAC-6, RAC-7 || Eng Team
|-
| TO-17 || Localization || RTL and strings in general (contextual warning) || Manual || RAC-8 || Eng Team
|}
|}


Line 113: Line 145:
! ID !! Description / Threat Description !! Covered by Test Objective !!  Magnitude !! Probability !! Priority !! Impact Score  
! ID !! Description / Threat Description !! Covered by Test Objective !!  Magnitude !! Probability !! Priority !! Impact Score  
|-
|-
| RAC-1 || Mouse and keyboard actions may not be recognized || TO-1 || 2-Moderate || 1-Unlikely || 3-High || 6
| RAC-1 || Refactoring of permissions prompt UI might affect Firefox base functionality  || TO-1 -> To-14 || 3-High|| 3-Almost Certain|| 3-High || 27
|-
| RAC-2 || Basic permission functionality might get broken || TO-1 -> To-14  || 3- High|| 3-Almost Certain|| 3-High || 27
|-
| RAC-3 || Mouse and keyboard actions may not be recognized || TO-13, TO-16 || 2-Moderate || 1-Unlikely || 3-High || 6
|-
| RAC-4 || Possible performance issues with multiple tabs for different permission prompt set || TO-15 || 2-Moderate || 2-Possible || 3-High || 12
|-
|-
| RAC-2 || Some Video types are not recognized and played || TO-3 || 3- High|| 2-Possible || 3-High || 18
| RAC-5 || Possible delay for the permission prompt when accessing a site || TO-14 || 3-High || 2-Possible || 3-High || 18
|-
|-
| RAC-3 || Possible performance issues with multiple tabs || TO-4 || 2-Moderate || 2-Possible || 3-High || 12
| RAC-6 || Backround colors and colors in general are visible and readable under high contrast  || TO-16  || 3-High || 3-Almost Certain|| 2-Medium || 18
|-
|-
| RAC-4 || After visiting for the first time a tab containing a video, the video starts with a delay || TO-1 || 3-High || 3-Almost Certain || 3-High || 27
| RAC-7 || Screen reader on the warning message? || TO-16 || 1- Low || 2-Possible || 3-High || 6
|-
|-
| RAC-8 || RTL  || TO-17  || 1- Low || 2-Possible || 2-Medium || 4
|}
|}


Line 132: Line 171:


'''Impact Score Breakdown:'''  
'''Impact Score Breakdown:'''  
* An impact value of 1 or 2 would describe an area which although should be covered there aren't expected any discoveries of critical issues.
* An impact value of 1, 2, 3, 4 would describe an area which although should be covered there aren't expected any discoveries of critical issues.
* An impact value of 4, 8 or 12 would describe an area in which we expect to find issues but those issues are not expected to be critical.
* An impact value of 6, 8, 9, 12 would describe an area in which we expect to find issues but those issues are not expected to be critical.
* An impact value of 18 or 27 would describe an area on which it is likely to find issues and those issues to be critical or blockers.
* An impact value of 18 or 27 would describe an area on which it is likely to find issues and those issues to be critical or blockers.


== Builds ==
== Builds ==
Line 151: Line 188:
|-
|-
| Start project  
| Start project  
|style="text-align:center;" | ||  
|style="text-align:center;" | 2017.01.04 ||  
|-
|-
| Study documentation/specs received from developers
| Study documentation/specs received from developers
|style="text-align:center;" |  ||  
|style="text-align:center;" |  2017.01.05
|style="text-align:center;" | 2017.01.12
|-
|-
| QA - Test plan creation  
| QA - Test plan creation  
|style="text-align:center;" |  ||  
|style="text-align:center;" |  2017.01.06
|style="text-align:center;" | 2017.01.13
|-
|-
| QA - Test cases/Env preparation  
| QA - Test cases/Env preparation  
|style="text-align:center;" | ||  
|style="text-align:center;" | 2017.01.10 ||  
|-
|-
| QA - Nightly Testing  
| QA - Nightly Testing  
|style="text-align:center;" | ||  
|style="text-align:center;" | 2017.01.12
|style="text-align:center;" | 2017.01.23
|-
|-
| QA - Aurora Testing
| QA - Aurora Testing
|style="text-align:center;" | ||
|style="text-align:center;" | 2017.01.30
|style="text-align:center;" | 2017.03.06
|-
|-
| QA - Beta Testing  
| QA - Beta Testing  
|style="text-align:center;" | ||  
|style="text-align:center;" | 2017.03.06
|style="text-align:center;" | 2017.03.31
|-
|-
| Release Date  
| Release Date  
Line 192: Line 234:
= Status =  
= Status =  
== Overview ==
== Overview ==
  Track the dates and build number where feature was released to Nightly
  First landed in Nightly 53, and we start testing 01/12/2017, with build [http://archive.mozilla.org/pub/firefox/nightly/2017/01/2017-01-12-03-03-01-mozilla-central/ Nightly link]
  Track the dates and build number where feature was merged to Aurora
  Landed on Aurora 53, on 01/23/2017, with build [http://archive.mozilla.org/pub/firefox/nightly/2017/01/2017-01-23-08-03-55-mozilla-aurora/ Aurora link]
  Track the dates and build number where feature was merged to Release/Beta
  Landed on Beta channel, with Beta 53 on 03/06/2017, with build [http://archive.mozilla.org/pub/firefox/candidates/53.0b1-candidates/ Beta 53 - b1]


== Risk analysis ==
Identify the high-risk assumptions
Identify existing bugs on the feature with high risk
Identify if other areas are affected by the fix
= References =
= References =
* Meta bug [https://bugzilla.mozilla.org/show_bug.cgi?id=1282768 1282768] - Convert permission notifications to use two buttons instead of the drop-down menu
* List and links for specs
* List and links for specs
   List and links for available specs - documents, user stories, specifications
   - [https://docs.google.com/document/d/143nEfWfIvFZD2-pFqE8GD85inFjFIhE2J8QeegQufc0/edit# Control Center V2 specs]
* Meta bug
  - [https://docs.google.com/document/d/1MaooflGxc2E9dcNVvz0AwYoTqySfEQOAhOpFNDsJBLs/edit# User research for Permission Prompt Redesign and Insecure Password UI]
   
  - [https://mozilla.invisionapp.com/share/9J7RMC2G7#/screens/170569580 feature samples]
* User Story Used
- [https://bugzilla.mozilla.org/show_bug.cgi?id=1188147 1188147] - [userstory] Notification of Single Permission
- [https://bugzilla.mozilla.org/show_bug.cgi?id=1188118 1188118] - [userstory] ID Block: Control Center "i" anchor icon
- [https://bugzilla.mozilla.org/show_bug.cgi?id=1188152 1188152] - [userstory] Notification when another request is made (multiple state)
- [https://bugzilla.mozilla.org/show_bug.cgi?id=1188355 1188355] - [userstory] CC: Site Permissions on main panel
  - [https://bugzilla.mozilla.org/show_bug.cgi?id=1188472 1188472] - [userstory] ID Block: Permission icon when blocked
 
= Testcases =  
= Testcases =  
== Overview ==
Summary of testing scenarios


== Test Areas ==
== Test Areas ==
Line 216: Line 259:
|-
|-
| Private Window  
| Private Window  
|style="text-align:center;" | ||  
|style="text-align:center;" | Yes ||  
|-
|-
| Multi-Process Enabled  
| Multi-Process Enabled  
|style="text-align:center;" | ||  
|style="text-align:center;" | Yes ||  
|-
|-
| Multi-process Disabled  
| Multi-process Disabled  
|style="text-align:center;" | ||  
|style="text-align:center;" | Yes ||  
|-
|-
| Theme (high contrast)  
| Theme (high contrast)  
|style="text-align:center;" | ||  
|style="text-align:center;" | Yes ||  
|-
|-
| '''UI'''  
| '''UI'''  
Line 231: Line 274:
|-
|-
| Mouse-only operation   
| Mouse-only operation   
|style="text-align:center;" | ||  
|style="text-align:center;" | Yes ||  
|-
|-
| Keyboard-only operation   
| Keyboard-only operation   
|style="text-align:center;" | ||
|style="text-align:center;" | Yes ||
|-
|-
| Display (HiDPI)  
| Display (HiDPI)  
|style="text-align:center;" | ||  
|style="text-align:center;" | Yes ||  
|-
|-
| Interraction (scroll, zoom)  
| Interraction (scroll, zoom)  
|style="text-align:center;" | ||  
|style="text-align:center;" | Yes ||  
|-
|-
| Usable with a screen reader   
| Usable with a screen reader   
|style="text-align:center;" | || e.g. with NVDA
|style="text-align:center;" | Yes || e.g. with NVDA
|-
|-
| Usability and/or discoverability testing   
| Usability and/or discoverability testing   
|style="text-align:center;" | || Is this feature user friendly
|style="text-align:center;" | Yes || Is this feature user friendly
|-
|-
| RTL build testing   
| RTL build testing   
|style="text-align:center;" | ||
|style="text-align:center;" | No||
|-
|-
| '''Help/Support'''  
| '''Help/Support'''  
Line 255: Line 298:
|-
|-
| Help/support interface required   
| Help/support interface required   
|style="text-align:center;" | || Make sure link to support/help page exist and is easy reachable.
|style="text-align:center;" | No || Make sure link to support/help page exist and is easy reachable.
|-
|-
| Support documents planned(written)   
| Support documents planned(written)   
|style="text-align:center;" | || Make sure support documents are written and are correct.
|style="text-align:center;" | No || Make sure support documents are written and are correct.


|-
|-
Line 265: Line 308:
|-
|-
| Feature upgrades/downgrades data as expected   
| Feature upgrades/downgrades data as expected   
|style="text-align:center;" | ||
|style="text-align:center;" | No ||
|-
|-
| Does sync work across upgrades   
| Does sync work across upgrades   
|style="text-align:center;" | ||  
|style="text-align:center;" | No ||  
|-
|-
| Requires install testing     
| Requires install testing     
|style="text-align:center;" | || separate feature/application installation needed (not only Firefox)
|style="text-align:center;" | No || separate feature/application installation needed (not only Firefox)
|-
|-
| Affects first-run or onboarding     
| Affects first-run or onboarding     
|style="text-align:center;" | || Florin/Lawrence are investigating if there is a dedicated QA for this, or we should test? Should be an yes/no and if is yes should add in detail column the team/person assigned.
|style="text-align:center;" | N/A || Florin/Lawrence are investigating if there is a dedicated QA for this, or we should test? Should be an yes/no and if is yes should add in detail column the team/person assigned.
|-
|-
| Does this affect partner builds? Partner build testing   
| Does this affect partner builds? Partner build testing   
|style="text-align:center;" | || yes/no options, add comment with details about who will lead testing
|style="text-align:center;" | N/A || yes/no options, add comment with details about who will lead testing


|-
|-
Line 284: Line 327:
|-
|-
| Enterprise administration   
| Enterprise administration   
|style="text-align:center;" | ||  
|style="text-align:center;" | N/A ||  
|-
|-
| Network proxies/autoconfig   
| Network proxies/autoconfig   
|style="text-align:center;" | ||  
|style="text-align:center;" | N/A ||  
|-
|-
| ESR behavior changes   
| ESR behavior changes   
|style="text-align:center;" | ||  
|style="text-align:center;" | N/A ||  
|-
|-
| Locked preferences   
| Locked preferences   
|style="text-align:center;" | ||
|style="text-align:center;" | N/A ||


|-
|-
Line 300: Line 343:
|-
|-
| Temporary or permanent telemetry monitoring   
| Temporary or permanent telemetry monitoring   
|style="text-align:center;" | || List of error conditions to monitor
|style="text-align:center;" | N/A || List of error conditions to monitor
|-
|-
| Telemetry correctness testing   
| Telemetry correctness testing   
|style="text-align:center;" | ||  
|style="text-align:center;" | N/A ||  
|-
|-
| Server integration testing   
| Server integration testing   
|style="text-align:center;" | ||  
|style="text-align:center;" | No ||  
|-
|-
| Offline and server failure testing   
| Offline and server failure testing   
|style="text-align:center;" | ||
|style="text-align:center;" | No ||
|-
|-
| Load testing   
| Load testing   
|style="text-align:center;" | ||
|style="text-align:center;" | No ||


|-
|-
Line 319: Line 362:
|-
|-
| Addon API required?   
| Addon API required?   
|style="text-align:center;" | ||  
|style="text-align:center;" | No ||  
|-
|-
| Comprehensive API testing   
| Comprehensive API testing   
|style="text-align:center;" | ||  
|style="text-align:center;" | No ||  
|-
|-
| Permissions   
| Permissions   
|style="text-align:center;" | ||  
|style="text-align:center;" | Yes ||  
|-
|-
| Testing with existing/popular addons
| Testing with existing/popular addons
|style="text-align:center;" | ||  
|style="text-align:center;" | No ||  


|-
|-
Line 335: Line 378:
|-
|-
| 3rd-party security review   
| 3rd-party security review   
|style="text-align:center;" | ||  
|style="text-align:center;" | No ||  
|-
|-
| Privilege escalation testing
| Privilege escalation testing
|style="text-align:center;" | ||  
|style="text-align:center;" | No ||  
|-
|-
| Fuzzing   
| Fuzzing   
|style="text-align:center;" | ||  
|style="text-align:center;" | No ||  


|-
|-
Line 348: Line 391:
|-
|-
| Testing against target sites   
| Testing against target sites   
|style="text-align:center;" | ||  
|style="text-align:center;" | Yes ||  
|-
|-
| Survey of many sites for compatibility   
| Survey of many sites for compatibility   
|style="text-align:center;" | ||  
|style="text-align:center;" | Yes ||  


|-
|-
Line 358: Line 401:
|-
|-
| Common protocol/data format with other software: specification available. Interop testing with other common clients or servers.   
| Common protocol/data format with other software: specification available. Interop testing with other common clients or servers.   
|style="text-align:center;" | ||  
|style="text-align:center;" | No ||  
|-
|-
| Coordinated testing/interop across the Firefoxes: Desktop, Android, iOS   
| Coordinated testing/interop across the Firefoxes: Desktop, Android, iOS   
|style="text-align:center;" | ||  
|style="text-align:center;" | No ||  
|-
|-
| Interaction of this feature with other browser features   
| Interaction of this feature with other browser features   
|style="text-align:center;" | ||  
|style="text-align:center;" | No ||  
|}
|}


== Test suite ==
== Test suite ==
  Full Test suite - Link with the gdoc, follow the format from [https://docs.google.com/spreadsheets/d/1N8dGK_uenhfcM9cEPC3HPHmi0jFN98FWtL2DuszrfLY/edit#gid=0 link]
  Full Test suite - in TestRail at [https://testrail.stage.mozaws.net/index.php?/suites/view/423&group_by=cases:section_id&group_order=asc link]
Smoke Test suite - Link with the gdoc, follow the format from [https://docs.google.com/spreadsheets/d/1N8dGK_uenhfcM9cEPC3HPHmi0jFN98FWtL2DuszrfLY/edit#gid=0 link]
Regression Test suite - Link with the gdoc - if available/needed.


= Bug Work =
= Bug Work =
Line 403: Line 444:
Check list
Check list
* All test cases should be executed
* All test cases should be executed
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)


Line 409: Line 449:
'''Nightly testing'''<br />
'''Nightly testing'''<br />


List of OSes that will be covered by testing<br />
*List of OSes that will be covered by testing<br />
** Windows 7 x64
** Ubuntu 16.04 x64
** Mac OS X 10.11 (El Capitan)
*Link for the tests run
*Link for the tests run
** Daily Smoke, use template from [https://docs.google.com/spreadsheets/d/1ch0OubyOWn1W-_JLF-38nvSvVB76Wmo1hkLccEtU5hk/edit?usp=sharing link]
** Full Test suite - [https://testrail.stage.mozaws.net/index.php?/runs/view/1949&group_by=cases:section_id&group_order=asc Nightly tests]
** Full Test suite, use template from [https://docs.google.com/spreadsheets/d/1ch0OubyOWn1W-_JLF-38nvSvVB76Wmo1hkLccEtU5hk/edit?usp=sharing link]
*This feature was treated as bug work, so no sign-off email was sent
** Regression Test suite, if needed/available


'''Merge to Aurora Sign-off'''
'''Merge to Aurora Sign-off'''<br />
List of OSes that will be covered by testing<br />
*List of OSes that will be covered by testing<br />
** Windows 10 x64
** Ubuntu 16.04 x64
** Mac OS X 10.12 (Sierra)
*Link for the tests run
*Link for the tests run
** Full Test suite
** Full Test suite - [https://testrail.stage.mozaws.net/index.php?/runs/view/2470&group_by=cases:section_id&group_order=asc Aurora tests]
*This feature was treated as bug work, so no sign-off email was sent
 
'''Merge to Beta/Pre-Release Sign-off'''
*List of OSes that will be covered by testing<br />
** Windows 10 x64
** Windows 7 x32
** Ubuntu 16.04 x64
** Mac OS X 10.12 (Sierra)
*Link for the tests run
** Full Test suite - [https://testrail.stage.mozaws.net/index.php?/runs/view/3335&group_by=cases:section_id&group_order=asc Beta test runs]


== Checklist ==
== Checklist ==
Line 426: Line 481:
|-
|-
|  Testing Prerequisites (specs, use cases)  
|  Testing Prerequisites (specs, use cases)  
| style="text-align:center;" |   
| style="text-align:center;" |  {{mdone|}}
| style="text-align:center;" |  
| style="text-align:center;" |  
|-
|-
|  Testing Infrastructure setup  
|  Testing Infrastructure setup  
|style="text-align:center;" |   ||  
|style="text-align:center;" | {{mdone|}}  ||  
|-
|-
|  Test Plan Creation  
|  Test Plan Creation  
| style="text-align:center;" |   ||  
| style="text-align:center;" | {{mdone|}} ||  
|-
|-
|  Test Cases Creation  
|  Test Cases Creation  
|style="text-align:center;" |   ||  
|style="text-align:center;" | {{mdone|}}  ||  
|-
|-
|  Full Functional Tests Execution  
|  Full Functional Tests Execution  
|style="text-align:center;" |   
|style="text-align:center;" |  {{mdone|}}
|style="text-align:center;" |   
|style="text-align:center;" |   
|-
|-
|  Automation Coverage ||
|  Automation Coverage  
|style="text-align:center;" |  
|style="text-align:center;" | {{mdone|}}
|style="text-align:center;" |
|-
|-
|  Performance Testing  
|  Performance Testing  
|style="text-align:center;" | ||  
|style="text-align:center;" | {{mdone|}} ||  
|-
|-
|  All Defects Logged || ||  
|  All Defects Logged  
|style="text-align:center;" | {{mdone|}}||  
|-
|-
|  Critical/Blockers Fixed and Verified || ||  
|  Critical/Blockers Fixed and Verified  
|style="text-align:center;" | {{mdone|}} ||  
|-
|-
|  Metrics/Telemetry||  
|  Metrics/Telemetry||  
|style="text-align:center;" |  
|style="text-align:center;" |  
|-
|-
|  QA Signoff  - Nightly Release||  
|  QA Signoff  - Nightly Release
|style="text-align:center;" | Email to be sent  
|style="text-align:center;" | N/A
|style="text-align:center;" | Was treated as bug work - no sign off email was sent.
|-
|-
|  QA Aurora - Full Testing  
|  QA Aurora - Full Testing  
|style="text-align:center;" | ||  
|style="text-align:center;" | {{mdone|}} ||  
|-
|-
|  QA Signoff  - Aurora Release ||  
|  QA Signoff  - Aurora Release  
|style="text-align:center;"| Email to be sent  
|style="text-align:center;" | N/A
|style="text-align:center;"| Was treated as bug work - no sign off email was sent.
|-
|-
|  QA Beta - Full Testing
|  QA Beta - Full Testing
|style="text-align:center;" | ||  
|style="text-align:center;" | {{mdone|}} ||  
|-
|-
|  QA Signoff  - Beta Release ||  
|  QA Signoff  - Beta Release ||  
|style="text-align:center;" | Email to be sent  
|style="text-align:center;" | Email to be sent  
|}
|}
Brindusa.tot
533

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/1158722...1167677"