QA/Pocket integration: Difference between revisions

From MozillaWiki
< QA
Jump to navigation Jump to search
(update related to bug verification, risk analysis and general documentation)
(→‎Sign off: expanded the sign off section to accommodate additional information)
Line 156: Line 156:
</div>
</div>


== Sign off ==
==Milestones==
===Firefox 38.0.5 Beta 1===
{| class="wikitable"
|-
| '''Builds''' || [url FTP], [url HTTP]
|-
| '''ID''' || ---
|-
| '''Changeset''' || [url rev] ([url pushlog])
|}
 
<big>'''Manual testing summary'''</big>
* '''Test coverage'''
** [url Smoke testing]
** Exploratory testing
* '''Reported bugs for this build'''
** [severity] Bug # - Bug_summary
* '''Test results'''
** [url MozTrap] Smoke test results
** Overall feature status after testing: ---
** Recommendation from QA: ---
 
==Sign off ==
===Criteria===
===Criteria===
* All the test cases were executed.
* All the test cases were executed.
Line 162: Line 184:


===Results===
===Results===
---
{{mok|}}''' Beta 38.0.5''' ''(yyyy-mm-dd)''

Revision as of 06:37, 6 May 2015

Desktop — Firefox Pocket Integration

Note: Please note that information is still being added to this page.

Ownership

Developer contacts: ---

QA Lead: Andrei Vaida, :avaida on IRC

Status

Overview

[LANDED] Nightly 40: first landed, disabled by default
[ON TRACK] Aurora 39: ---
[ON TRACK] Beta 38: ---
[ON TRACK] Release 38: ---

Timeline

What? Expected Actual
Prototype landing on m-c     Wed, April 29th       ---
Uplift to m-b ---    ---
GTB: Beta 1 Thu, May 7th    ---
GTB: RC1 Thu, may 21st    ---

Risk analysis

Event Probability Impact Current situation Mitigation steps
Users that have the Pocket add-on already installed are not handled properly after updating to a Pocket-integrated Firefox build. probable high --- Periodical regression testing covers this area.
Users that have the Pocket social media service already installed are not handled properly after updating to a Pocket-integrated Firefox build. probable high --- Periodical regression testing covers this area.
User fails to log into Pocket using the FxA sign-in flow. probable high --- Periodical regression testing covers this area.
Page fails to be saved to Pocket. probable high --- Periodical regression testing covers this area.
Saved page fails to be opened from Pocket. probable high --- Periodical regression testing covers this area.
Reader View fails to display pages saved to Pocket. probable high --- Periodical regression testing covers this area.

Summary

Context

The features made available by this integration:

  • Save to Pocket button in the toolbar
  • Open Pocket item from the bookmarks menu
  • Save to Pocket button in the context menu
  • FxA sign-in for Pocket
  • Save to Pocket button from Reader View's controls
  • UI tour
  • Reader View aesthetic to align with what Pocket is currently offering

Enabling the feature

Pocket is available starting with Firefox 40 and it can be enabled by setting the following pref to true:

browser.pocket.enabled

A hostname is also required to ensure Pocket's functionality, that's being done through the following pref:

browser.pocket.hostname

References

  • The Pocket toolbar panel mock is available here.
  • A design mock depicting a few Pocket flows is available here
  • A design mock showing confirmation panel's variations and states is available here.

Test cases

Overview

Targeted scenarios:

Click the expand button to see a few of the targeted user stories

  • As a user, I can log into Pocket with an existing Firefox Account, so that I can start using the feature without having to create a new set of credentials.
  • As a user, I can log into Pocket with an existing Pocket account, so that I can start using the feature without having to create a new set of credentials.
  • As a user, I can log into Pocket with a new Firefox Account, so that I can start using Pocket.
  • As a user, I can log into Pocket with a new Pocket account, so that I can start using Pocket.
  • As a user, I can save a page to Pocket by clicking the dedicated Pocket button from the Location Bar.
UPDATE 2015-04-28: there won't be a dedicated Pocket button in the Location Bar, at least for now
  • As a user, I can save a page to Pocket by clicking the dedicated Pocket button from the controls available in Reader View.
  • As a user, I can save a page to Pocket by clicking the dedicated Pocket option from the context menu.
  • As a user, I can view the items saved to Pocket from the dedicated Pocket entry available in the bookmarks menu.
  • As a user, I can start using Pocket on an older profile that already has it installed as an add-on.
  • As a user, I can start using Pocket on an older profile that already has it installed via Social API.
  • As a user, I can remove a page from Pocket using the dedicated Pocket button from the Location Bar.
UPDATE 2015-04-28: there won't be a dedicated Pocket button in the Location Bar, at least for now
  • As a user, I can remove a page from Pocket using the dedicated Pocket entry from the bookmarks menu.
  • As a user, I can remove a page from Pocket using the dedicated Pocket option from the context menu.
UPDATE 2015-04-28: the only option available in the context menu will be to save a page to Pocket, NOT to remove it
  • As a user, I can be notified in a friendly way if my connection or the Pocket servers are down.

Draft test suites:

MozTrap test suites:

  • [ON TRACK] Smoke Test Suite
  • [ON TRACK] Regression Test Suite

Bug work

Tracking bugs

[META] Bug 1155467 - [meta] Firefox Pocket integration

[META] Bug 1157495 - [meta] Pocket quality and test eng tasks

[META] Bug 1157492 - [meta] Pocket integration metrics tracking

Bug triage +/- for verification

[QE-VERIFY+] Bug 1155523 - Implement rough first-pass at Pocket toolbar button UI
[QE-VERIFY+] Bug 1155517 - Change Reader View to have a "Save Page to Pocket" button instead of "Add To Reader List"
[QE-VERIFY+] Bug 1155518 - Implement "Save to Pocket" context menu item
[QE-VERIFY+] Bug 1155519 - Add "View Pocket Items" menuitem to the bookmarks menu
[QE-VERIFY-] Bug 1155520 - Add telemetry for Pocket integration
[QE-VERIFY-] Bug 1158289 - Optimize Reader View's line length to have between 45 and 75 characters
[QE-VERIFY+] Bug 1158281 - Match Pocket's Reader View Sepia Theme

Bug fix verification

[VERI] Bug 1155523 - Implement rough first-pass at Pocket toolbar button UI

2015-04-30: partial functionality confirmed fixed on 40.0a1, across platforms

[VERI] Bug 1156878 - Send a request to the server when clicking the Pocket toolbar button

2015-04-30: confirmed fixed on 40.0a1, across platforms
Logged bugs

Bug 1161502 - The question mark icon next to the "Sign Up with Firefox" button seems misaligned

2015-05-05: found while performing functional testing on 40.0a1

Bug 1161504 - The "Save" button from the Pocket panel appears to have a visited styling applied

2015-05-05: found while performing functional testing on 40.0a1

Bug 1161508 - The panel shown by the Pocket button after saving a page displays top and bottom borders

2015-05-05: found while performing functional testing on 40.0a1

Bug 1161510 - main.js:481:12 throws an error in the Browser Console when "Sign Up with Firefox" is clicked

2015-05-05: found while performing functional testing on 40.0a1

Bug 1161514 - The sign-up panel shown by the Pocket button displays top and bottom borders on Mac and Windows

2015-05-05: found while performing functional testing on 40.0a1

Bug 1161518 - The user is able to drag and drop the links from the Pocket panel

2015-05-05: found while performing functional testing on 40.0a1

Bug 1161522 - The user is able to save Pocket's "My List" to itself, twice

2015-05-05: found while performing functional testing on 40.0a1

Milestones

Firefox 38.0.5 Beta 1

Builds [url FTP], [url HTTP]
ID ---
Changeset [url rev] ([url pushlog])

Manual testing summary

  • Test coverage
    • [url Smoke testing]
    • Exploratory testing
  • Reported bugs for this build
    • [severity] Bug # - Bug_summary
  • Test results
    • [url MozTrap] Smoke test results
    • Overall feature status after testing: ---
    • Recommendation from QA: ---

Sign off

Criteria

  • All the test cases were executed.
  • All the blocker, critical, major bugs have been fixed.

Results

[ON TRACK] Beta 38.0.5 (yyyy-mm-dd)