QA/Push Notifications

Test Plan (Overview)

SUMMARY

• The purpose of this wiki is to serve as a general testplan for verifying desktop Push notifications.

OWNERSHIP

• Manual tests will be defined / outlined and executed by the Cloud Services QA Team
• Manual testing will be handed off to Firefox Quality Engineering Team
• Automated E2E smoke tests will be created by the Cloud Services QA Team
• Any further feature automation is TBD (by Firefox Quality Engineering Team)

TEST RESULTS

• Test results will be posted in Bugzilla (component: TBD).

SIGNOFF CRITERIA Cloud Services QA team will verify that:

• Desktop push notifications are working properly in FF Nightly.
• Interaction between push-service (autopush[3]), ServiceWorkers[1], and desktop UI is working as expected.

Testplan (Overview)

REFERENCE

• [1] http://www.w3.org/TR/service-workers/
• [2] http://www.w3.org/TR/push-api/
• [3] autopush: https://github.com/mozilla-services/autopush/
• [4] push-service: https://github.com/mozilla-services/push-service

Test Cases

NOTIFICATIONS API

Common

Shared steps between Notifications tests:

setup()

1. Open https://pdehaan.github.io/push-notification-test/
2. Click pop Notification button.

teardown()

1. Click on lock icon in the address bar.
2. Set Show Notifications permission to "Always Ask".

Tests

Name: Doorhanger default state test
Description: Verify the doorhanger reports if the site has requested any notification permissions.
Steps:

1. Click the lock icon in the address bar.
2. Verify the permissisons section says "You have not granted this site any special permissions."

Name: TBD
Description: Verify that trying to display a notification causes a dialog to appear.
Steps:

1. setup()
2. Click the "close" button in the upper right corner of the notifications doorhanger.
3. Click "pop Notification" button.
4. Verify the notification doorhanger appears again and requests permissions.

Name: TBD
Description: Verify clicking outside of a notification dismisses it automatically (OS X only).
Steps:

1. setup()
2. Click outside of the notifications doorhanger.
3. Verify the notifications doorhanger is dismissed.

Name: TBD
Description: Verify clicking on the "Learn more" link opens the target page.
Steps:

1. setup()
2. Click the "Learn more..." link in the notifications doorhanger.
3. Verify the browser redirects to https://www.mozilla.org/en-US/firefox/push/ (in a new tab).

Name: TBD
Description: Verify that a notification is auto-closed after X seconds.
Steps:

1. setup()
2. Select "Always Show Notifications" from the notifications doorhanger.
3. Verify the notification appears and auto-dismisses after X seconds.
4. teardown()

Name: TBD
Description: Verify that blocking notifications prevents notifications from displaying.
Steps:

1. setup()
2. Select "Always Block Notifications" from the notifications doorhanger.
3. Verify no notification appears.
4. Click "pop Notification" button.
5. Verify no notification appears.
6. teardown()

Name: TBD
Description: Verify clicking "Not Now" from the notification permissions doorhanger causes the permissions popup to appear again.
Steps:

1. setup()
2. Select "Not Now" from the notifications doorhanger.
3. Verify no notification appears.
4. Click "pop Notification" button.
5. Verify notification doorhanger appears again.
6. teardown()

Name: TBD
Description: Verify changing notification permissions in the address bar causes the notification permissions screen to reappear.
Steps:

1. setup()
2. Select "Always Show Notifications" from the notifications doorhanger.
3. Verify notification appears and auto-dismisses after X seconds.
4. Click the lock button in the address bar.
5. Change notification permissions to "Always Ask".
6. Click "pop Notification" button.
7. Verify notification doorhanger appears again.
8. Select "Always Show Notifications" from the notifications doorhanger.
9. Verify notification appears.
10. Click the lock button in the address bar.
11. Change notification permissions to "Block".
12. Click "pop Notification" button.
13. Verify no notification appears.
14. Click the lock button in the address bar.
15. Change notification permissions to "Allow".
16. Click "pop Notification" button appears.
17. Verify notification appears.
18. teardown()

Name: TBD
Description: Verify programmatically closing a notification causes it to immediately disappear.
Steps:

1. setup()
2. Select "Always Show Notifications" from the notifications doorhanger.
3. Verify notification appears.
4. Click "closeNotification" button.
5. Verify notification is immediately dismissed.
6. teardown()

Name: TBD
Description: Verify notification can be manually dismissed by swiping it (OS X only).
Steps:

1. setup()
2. Select "Always Show Notifications" from the notifications doorhanger.
3. Click and drag notification to dismiss it.
4. teardown()

Name: TBD
Description: Verify notification can be dismissed by clicking the close button.
Steps:

1. setup()
2. Select "Always Show Notifications" from the notifications doorhanger.
3. Hover over the doorhanger and click the "Close" button.
4. Verify notification is immediately dismissed.
5. teardown()

Name: TBD
Description: Verify setting `requireInteraction` to true causes the notification to remain open until explicitly closed (OS X only).
Steps:

1. setup()
2. Select "Always Show Notifications" from the notifications doorhanger.
3. Set "requireInteraction" dropdown to `true`.
4. Click "pop Notification" button.
5. Verify that the notification persists until explicitly closed.
6. teardown()

Name: TBD
Description: Verify setting `requireInteraction` to false (default) causes the notification to auto-close after X seconds.
Steps:

1. setup()
2. Select "Always Show Notifications" from the notifications doorhanger.
3. Set "requireInteraction" dropdown to `false`.
4. Click "pop Notification" button.
5. Verify the notification auto dismisses after X seconds.
6. teardown()

Name: TBD
Description: Verify clicking on a notification opens a specified URL.
Steps:

1. setup()
2. Select "Always Show Notifications" from the notifications doorhanger.
3. Click on the notification.
4. Verify that https://www.mozilla.org/en-US/ opens in a new tab.
5. teardown()

Name: TBD
Description: Verify setting target opens specified URL in the specified window.
Steps:

1. Open https://pdehaan.github.io/push-notification-test/
2. Set "target" to `_top`.
3. Click "pop Notification" button.
4. Select "Always Show Notifications" from the notifications doorhanger.
5. Verify that https://www.mozilla.org/en-US/ opens in the current tab.
6. teardown()

Name: TBD
Description: Verify setting a target causes a notification to open in the same window.
Steps:

1. Open https://pdehaan.github.io/push-notification-test/
2. Set "target" to `foobar`.
3. Click "pop Notification" button.
4. Select "Always Show Notifications" from the notifications doorhanger.
5. Verify that https://www.mozilla.org/en-US/ opens in a new tab.
6. Click "pop Notification" button.
7. Verify that https://www.mozilla.org/en-US/ opens in the same tab as step 5.
8. teardown()

Name: TBD
Description: Verify setting a _blank target causes a notification to open in the new window.
Steps:

1. Open https://pdehaan.github.io/push-notification-test/
2. Set "target" to `_blank` (default).
3. Click "pop Notification" button.
4. Select "Always Show Notifications" from the notifications doorhanger.
5. Verify that https://www.mozilla.org/en-US/ opens in a new tab.
6. Click "pop Notification" button.
7. Verify that https://www.mozilla.org/en-US/ opens in a new tab.
8. teardown()

Name: TBD
Description: Verify clicking the pop Notification button causes the notification to display multiple times.
Steps:

1. setup()
2. Select "Always Show Notifications" from the notifications doorhanger.
3. Click "pop Notification" button 10 times.
4. Verify the notification only appears once (and says "9 new notifications").
5. Verify there are 11 notifications in the OSX notifications tray.
6. teardown()

Name: TBD
Description: Verify the notification is not dismissed when hovering over the notification.
Steps:

1. setup()
2. Select "Always Show Notifications" from the notifications doorhanger.
3. Hover over the notification and make sure the notification is not auto-dismissed.
4. Move mouse off of notification.
5. Verify that the notification disappears after X seconds.
6. teardown()

Name: TBD
Description: Verify the notification is not dismissed when hovering over an item in the submenu (OS X only).
Steps:

1. setup()
2. Select "Always Show Notifications" from the notifications doorhanger.
3. Hover over the notification and click the "..." button in the lower-right corner.
4. Click "Disable notifications from this site" menu option.
5. Click "pop Notification" button.
6. Verify notification doorhanger appears.
7. teardown()

Name: TBD
Description: Verify clicking Notification Settings in the notification causes the preferences page to appear.
Steps:

1. setup()
2. Select "Always Show Notifications" from the notifications doorhanger.
3. Hover over the notification and click the "..." button in the lower-right corner.
4. Click "Notification settings" menu option.
5. Verify that about:preferences#content opens in a new tab.
6. teardown()

Name: TBD
Description: Verify allowing notifications reports correctly in about:preferences#content
Steps:

1. setup()
2. Select "Always Show Notifications" from the notifications doorhanger.
3. Open about:preferences#content
4. Click "Choose" button.
5. Verify that https://pdehaan.github.io/ appears in site list and it's status is set to "Allow".
6. teardown()

Name: TBD
Description: Verify blocking notifications reports correctly in about:preferences#content
Steps:

1. setup()
2. Select "Always Block Notifications" from the notifications doorhanger.
3. Open about:preferences#content
4. Click "Choose" button.
5. Verify that https://pdehaan.github.io/ appears in site list and it's status is set to "Block".
6. teardown()

Name: TBD
Description: Verify a long "Title" doesn't cause the notification to resize.
Steps:

1. Open https://pdehaan.github.io/push-notification-test/
2. Set "Title"to a a long string.
3. Click "pop Notification" button.
4. Verify the title text truncates and doesn't resize notification popup.
5. teardown()

Name: TBD
Description: Verify a long "Body Text" doesn't cause the notification to resize.
Steps:

1. Open https://pdehaan.github.io/push-notification-test/
2. Set "Body Text"to a a long string.
3. Click "pop Notification" button.
4. Verify the body text truncates and doesn't resize notification popup.
5. teardown()

PUSH API

Common

Shared steps between Push API tests:

setup()

1. Open https://pdehaan.github.io/push-notification-test/
2. Click "register Service Worker" button.

teardown()

1. Click "Unregister Service Worker" button.

Tests

Name: TBD
Description: Verify registering a service worker causes it to appear in about:serviceworkers.
Steps:

1. setup()
2. Verify HTML output is:

   registering service worker

   registered service worker. scope: https://pdehaan.github.io/push-notification-test/

3. Click "Check Service Worker" button.
4. Verify HTML output is:

   checking service worker

   Service worker active

5. Open about:serviceworkers
6. Verify service worker is registered to https://pdehaan.github.io/ origin.
7. Go back to https://pdehaan.github.io/push-notification-test/
8. Click "Unregister Service Worker" button.
9. Open about:serviceworkers
10. Verify "No Service Workers registered." message (refresh page if it was previously open tab).
11. teardown()

Name: TBD
Description: Verify sending a message to a service worker causes it to echo the message.
Steps:

1. setup()
2. Click "subscribe to push" button.
3. Click "sendMsgToSW" button.
4. Verify "messageChannel.port1.onmessage: SW echo: HelloWorld" output on page.
5. Change text box value to "999".
6. Click "sendMsgToSW" button.
7. Verify "messageChannel.port1.onmessage: SW echo: 999" output on page.
8. teardown()

PUSH API - REFERENCE

Push WebSocket Endpoints:

source:

https://mana.mozilla.org/wiki/display/SVCOPS/SimplePush#SimplePush-Deployments

PUSH ENDPOINTS - STAGE:

Production:	wss://push.services.mozilla.com/
Production:	wss://updates.push.services.mozilla.com/
Staging:	wss://autopush.stage.mozaws.net/
Staging:	wss://updates-autopush.stage.mozaws.net/
Dev:	wss://autopush.dev.mozaws.net/

STATUS / HEALTH:

Production:	https://updates.push.services.mozilla.com/health
Production:	https://updates.push.services.mozilla.com/status
Staging:	https://updates-autopush.stage.mozaws.net/health

CONTACTS

• pdehaan@mozilla
• rpappalardo@mozilla
• ryanvm@mozilla