QA/Push Notifications: Difference between revisions

Jump to navigation Jump to search

QA/Push Notifications (view source)

Revision as of 18:14, 7 January 2016

5,458 bytes added ,  7 January 2016
→‎Test Plan (Overview) https://irccloud.mozilla.com/file/8eTYehOd/hot.gif: mfunches updates for Push 1.0
m (Canonical name for the category is "Test Plans," not "Test Paan.")
Line 5: Line 5:


'''SUMMARY'''
'''SUMMARY'''
* The purpose of this wiki is to serve as a general testplan for verifying desktop Push notifications.
* The purpose of this wiki is to serve as a general test plan for verifying desktop Push notifications.




'''OWNERSHIP'''
'''OWNERSHIP'''
* Manual tests will be defined / outlined and executed by the Cloud Services QA Team
* Manual tests will be defined / outlined and executed by the Cloud Services QA Team
* Manual testing will be handed off to sOFTVISION qa tEAM
* Manual testing will be handed off to Softvision QA Team
* Automated E2E smoke tests will be created by the Cloud Services QA Team
* Automated E2E smoke tests will be created by the Cloud Services QA Team
* Any further feature automation is TBD (by Firefox Quality Engineering Team)
* Any further feature automation is TBD (by Firefox Quality Engineering Team)
Line 19: Line 19:




'''SIGNOFF CRITERIA'''
'''SIGN-OFF CRITERIA'''
Cloud Services QA team will verify that:
Cloud Services QA team will verify that:
* Desktop push notifications are working properly in FF Nightly.
* Desktop push notifications are working properly in FF Nightly.
* Interaction between push-service (autopush[3]), ServiceWorkers[1], and desktop UI is working as expected.
* Interaction between push-service (auto-push[3]), Service Workers[1], and desktop UI is working as expected.


== Test Objectives ==
Following table contains main features to be tested:
{| class="wikitable"
|-
! Ref !! Function !! Test Objective !! Evaluation Criteria
|-
| 1 || Notification - basic scenarios || Pop-up notifications display or not based on permissions and settings || Notifications display or disable accordingly.
|-
| 2 || Multi-Process (e10) Enable & Disabled, Private Window || Validate that Notifications display for all window types || Multi-Process (e10) and Private windows display notifications
|}
== Test Types ==
{|class="wikitable" style="width:60%"
|-
! Type of Testing !! Required (Yes/No) !! Owners !! Details/Comments
|-
| Manual
|style="text-align:center;" | Yes
|style="text-align:center;" | Moz QA Eng Team LV||
|-
| Automation
|style="text-align:center;" | No
|style="text-align:center;" |  ||
|-
| Performance
|style="text-align:center;" | No ||  ||
|-
| Security
|style="text-align:center;" | No ||  ||
|-
| Usability
|style="text-align:center;" | No ||  ||
|-
| Metrics/Telemetry
|style="text-align:center;" | No ||  ||
|}
== Builds ==
This section should contain links for builds with the feature -


'''Testplan (Overview)'''
Nightly builds with the search refactor fix can be reached at [http://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central/ link]
Platform(s) Windows 10, Windows 8, Windows 8.1, Windows Vista, Windows 7, Windows XP, Ubuntu 14.04, Mac OS 10
 
== Test Execution Schedule ==
The following table identifies the anticipated testing period available for test execution.
{| class="wikitable" style="width:60%"
|-
! Project phase !! Start Date !! End Date
|-
| Start project
|style="text-align:center;" | 10.23.2015 ||
|-
| Study documentation/specs received from developers
|style="text-align:center;" | 10.23.2015 ||  -
|-
| QA - Test plan creation
|style="text-align:center;" | 10.23.2015 ||
|-
| QA - Test cases/Env preparation
|style="text-align:center;" | 10.23.2015  || -
|-
| QA - Nightly Testing
|style="text-align:center;" | 10.23.2015 || 12.16.2015
|-
| QA - Aurora Testing
|style="text-align:center;" |  || 
|-
| QA - Beta Testing
|style="text-align:center;" |  ||
|-
| Release Date
|style="text-align:center;" | ||
|}
 
== Testing Tools ==
Detail the tools to be used for testing, for example see the following table:
{| class="wikitable" style="width:60%"
|-
! Process !! Tool
|-
| Test plan creation || Mozilla wiki
|-
| Test case creation || Google docs
|-
| Test case execution || Google docs
|-
| Bugs management || Bugzilla
|}
 
= Test cases =
== Overview ==
Platform(s) Windows 10, Windows 8, Windows 8.1, Windows Vista, Windows 7, Ubuntu 14.04, Mac OS X 10


Daily Smoke Test will test the following: [Cycle>Check Enable multi-process Nightly, Unchecked Enable multi-process, Nightly Private Window]
Daily Smoke Test will test the following: [Cycle>Check Enable multi-process Nightly, Unchecked Enable multi-process, Nightly Private Window]
Line 34: Line 123:
! Test Name !! Description !!  
! Test Name !! Description !!  
|-
|-
| Notifications Dialog || Verify that trying to display a notification causes a dialog to appear.
| Notification Dialog || Verify that trying to display a notification causes a dialog to appear. ||
|-
|-
| Notifications Dialog|| Verify that trying to display a notification causes a dialog to appear.
| Notification Permissions|| Verify that Permissions can be set & adhered to [Always Ask, Allow, Block] . ||
|-
|-
| Notification: Timeout|| Verify that a notification is auto-closed after X seconds.
| Notification: Timeout|| Verify that a notification is auto-closed after X seconds. ||
|-
|-
| Notification: Blocking|| Verify that blocking notifications prevents notifications from displaying.
| Notification: Blocking|| Verify that blocking notifications prevents notifications from displaying. ||
|-
|-
| Notification: Not Now|| Verify clicking "Not Now" from the notification permissions doorhanger causes the permissions popup to appear again.
| Notification: Not Now|| Verify clicking "Not Now" from the notification permissions door-hanger causes the permissions popup to appear again. ||
|-
|-
| Notification: Close Button|| Verify notification can be dismissed by clicking the close button.
| Notification: Close Button|| Verify notification can be dismissed by clicking the close button. ||
|-
|-
| Notification: Redirection|| Verify clicking on a notification opens a specified URL.
| Notification: Redirection|| Verify clicking on a notification opens a specified URL. ||
|-
|-
| Notification: Pop Up Multi Message || Verify clicking the pop Notification button causes the notification to display multiple times.
| Notification: Pop Up Multi Message || Verify clicking the pop Notification button causes the notification to display multiple times. ||
|-
|-
|}
|}
== Test Areas ==
{| class="wikitable" style="width:60%"
|-
! Test Areas !! Covered !! Details
|-
| Private Window
|style="text-align:center;" | {{mdone|}} ||
|-
| Multi-Process Enabled
|style="text-align:center;" | {{mdone|}} ||
|-
| Multi-process Disabled
|style="text-align:center;" | {{mdone|}} ||
|-
| Display (HiDPI)
|style="text-align:center;" | N/A || 
|-
| Interaction (scroll, zoom)
|style="text-align:center;" | N/A ||
|-
| Theme (high contrast)
|style="text-align:center;" | N/A ||
|}
== Test suites ==
Full Test Suite - can be reached at [https://docs.google.com/spreadsheets/d/1Gy2NYVkiPJOwK4v-L5TrM_Am3T-ZeQkVAN7mgSwXyj4/edit#gid=1004672113/edit?usp=sharing link ]
= Bugs Work =
Bug information can be displayed at [https://docs.google.com/spreadsheets/d/1Gy2NYVkiPJOwK4v-L5TrM_Am3T-ZeQkVAN7mgSwXyj4/edit#gid=1004672113/edit?usp=sharing link ]
= Sign off  =
== Criteria ==
* All test cases should be executed
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)
== Results ==
'''Nightly testing 45.0a1'''
* [Complete Push 1.0] Full Test  Suite results [https://docs.google.com/spreadsheets/d/1Gy2NYVkiPJOwK4v-L5TrM_Am3T-ZeQkVAN7mgSwXyj4/edit#gid=1004672113/edit?usp=sharing ]<br />
'''Beta testing 44.0b4'''
* [Complete Push 1.0] Full Test  Suite results [https://docs.google.com/spreadsheets/d/1Gy2NYVkiPJOwK4v-L5TrM_Am3T-ZeQkVAN7mgSwXyj4/edit#gid=1004672113/edit?usp=sharing ]<br />
** Manual testing covered Windows 7 x64, Windows 8 x64, Windows 8.1 x64  Windows 10 x64, Windows Vista x32, Ubuntu 14.04 x32, Mac OS x 10.11
<br />
== Checklist ==
{| class="wikitable" style="width:60%"
|-
! Exit Criteria !! Status !! Notes/Details
|-
|  Testing Prerequisites (specs, use cases)
| style="text-align:center;" | {{mdone|}} 
| style="text-align:center;" |
|-
|  Testing Infrastructure setup
|style="text-align:center;" |  {{mdone|}} ||
|-
|  Test Plan Creation
| style="text-align:center;" |  {{mdone|}} ||
|-
|  Test Cases Creation
|style="text-align:center;" |  {{mdone|}} ||
|-
|  Full Functional Tests Execution
|style="text-align:center;" |  {{mdone|}}
|style="text-align:center;" | 
|-
|  Performance Testing
|style="text-align:center;" | N/A ||
|-
|  All Defects Logged
|style="text-align:center;"| {{mdone|}} ||
|-
|  Critical/Blockers Fixed and Verified
|style="text-align:center;"| {{mdone|}} ||
|-
|  Daily Status Report (email/etherpad statuses/ gdoc with results)
|style="text-align:center;"| {{mdone|}} ||
|-
|  Metrics/Telemetry
|style="text-align:center;" |  N/A  ||
|-
|  UAT Test Cases (signoff By Users)
|style="text-align:center;" |  N/A  ||
|-
|  QA Signoff  - Nightly Release
|style="text-align:center;" | {{mdone|}}
|style="text-align:center;" | Email sent Jan 04, 2016 [Jan 06, 2016]
|-
|  QA Postmortem
|style="text-align:center;" | {{mdone|}}
|style="text-align:center;" | [https://public.etherpad-mozilla.org/p/r.6041c8cb96bb188c57c385daaa5c38a9 Meeting Notes]
|}


'''REFERENCE'''
'''REFERENCE'''
Mfunches
91

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/1111778"

Navigation menu