MozillaWiki

Add-ons/QA/Testplan/Improve Storage Local API: Difference between revisions

Page Discussion

Add-ons/QA/Testplan/Improve Storage Local API (view source)

Revision as of 08:40, 22 May 2018

8,689 bytes added ,  22 May 2018
no edit summary
(Created blank page)
 
No edit summary
Line 1: Line 1:
'''Revision History'''


{| class="wikitable" style="width:60%"
|-
! Date !! Version !! Author !! Description
|-
| 05/22/2018 || 1.0 || Cosmin Badescu || Created first draft
|-
|}
= Overview =
The purpose of this feature is to check that the migration from storage.local JSONFile backend to storage.local IndexedDB backend works correctly.
== Purpose ==
This document purports to detail the test approach for the improvements of Local Storage API  and includes Entry/Exit criteria, Scope for testing, links to test cases etc
=== Entry Criteria ===
* QA has access to all the PRDs, mocks and related documents
* The feature has landed on Nightly
* AMO parts has landed on dev
=== Exit Criteria ===
* All the bugs against the feature have been triaged
* All the P1/P2 bugs have been fixed
* All the resolved bugs have been verified by QA
* The find/fixed rate is going down over a predefined period of time
=== Acceptance Criteria ===
This section broadly outlines when the product is ready to ship
* QA has signed off
* All the required Telemetry is in place
* All info is localized at least for a pre-defined set of locales
* All the necessary PR/blogposts have been sent out
== Scope ==
This section describes what parts of the feature will be tested and what parts won't be.
===what's in scope?===
* The behavior of the storage.local JSONFile backend and storage.local IndexedDB backend when the extensions use them.
===what's out of scope?===
* Security testing
* Performance testing
== Ownership ==
Dev Lead: [mailto:lgreco@mozilla.com Luca Greco]; irc nick :rpl <br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br />
= Requirements for testing =
== Environments ==
OSes covered: Windows, Mac OS X, Linux
== Channel dependent settings (configs) and environment setups ==
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
=== Nightly ===
<div class="mw-collapsible-content">
*Enable extensions.webextensions.useExtensionStorageIDB from about:config
</div>
=== Beta ===
<div class="mw-collapsible-content">
*Enable extensions.webextensions.useExtensionStorageIDB from about:config
</div>
=== Release ===
<div class="mw-collapsible-content">
The feature is enabled by default
</div>
</div>
= Test Strategy =
== Test Objectives ==
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.
This could be documented in bullet form or in a table similar to the one below.
{| class="wikitable"
|-
! Ref !! Function !! Test Objective !! Test Type !! Owners
|-
| TO-1 || Installing from AMO || To verify that the extension uses the API correctly || Manual || Add-ons QA Team
|-
| TO-2 || Installing from local files || To verify that the extension uses the API correctly || Manual || Add-ons QA Team
|-
| TO-3 || Add-on updates || To verify that the extension uses the API correctly || Manual || Add-ons QA Team
|-
| TO-4 || Sideloading || To verify that the extension uses the API correctly || Manual || Add-ons QA Team
|}
== Builds ==
This section should contain links for builds with the feature -
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds
== Test Execution Schedule ==
The following table identifies the anticipated testing period available for test execution.
{| class="wikitable" style="width:60%"
|-
! Project phase !! Start Date !! End Date
|-
| Start project
|style="text-align:center;" |  ||
|-
| Study documentation/specs received from developers
|style="text-align:center;" |  ||
|-
| QA - Test plan creation
|style="text-align:center;" | 05-22-2018 ||
|-
| QA - Test cases/Env preparation
|style="text-align:center;" | 05-21-2018 || N/A
|-
| QA - Nightly Testing
|style="text-align:center;" |  ||
|-
| QA - Beta Testing
|style="text-align:center;" |  ||
|-
| Release Date
|style="text-align:center;" | ||
|}
== Testing Tools ==
Detail the tools to be used for testing, for example see the following table:
{| class="wikitable" style="width:50%"
|-
! Process !! Tool
|-
| Test plan creation || Mozilla wiki
|-
| Test case creation || [ Docs] / [ TestRail]
|-
| Test case execution || [ Docs] / [ TestRail]
|-
| Bugs management || Bugzilla / Github
|}
= Status =
== Overview ==
Track the dates and build number where feature was released to Nightly
Track the dates and build number where feature was merged to Release/Beta
== Risk analysis ==
Identify the high-risk assumptions
Identify existing bugs on the feature with high risk
Identify if other areas are affected by the fix
= References =
* List and links for specs
  PRD - TBD
  Install flow - [https://docs.google.com/document/d/1IDNHoneiARuw8PT6KIQGp03WxFmaQYijpluZA0C-E7I/edit Link]
 
* {{Bug|1406181}} - Use indexedDB as the backend for storage.local
<bugzilla>
    {
        "blocks":[1406181],
        "include_fields": "id, priority, component, assigned_to, summary, status, target_milestone"
    }
</bugzilla>
= Testcases =
== Overview ==
Summary of testing scenarios
== Test Areas ==
{| class="wikitable" style="width:80%"
|-
! Test Areas !! Covered !! Details
|-
| '''Installing from AMO'''
|style="text-align:center;" | ||
|-
| '''Installing from local files'''
|style="text-align:center;" |  ||
|-
| '''Installing from thirdparty'''
|style="text-align:center;" |  ||
|-
| '''Add-on updates'''
|style="text-align:center;" |  ||
|-
| '''Sideloading'''
||  ||
|-
| '''Other''' 
|style="text-align:center;" |  ||
|-
|}
== Test suite ==
* Link for the [ Initial test planning]
* Link for the [ Google doc tests]
* Link for the [https://testrail.stage.mozaws.net/index.php?/suites/view/1735&group_by=cases:section_id&group_order=asc TestRail tests]
= Bug Work =
Tracking bug - []
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
====== Bug fix verification ======
<div class="mw-collapsible-content">
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Display permissions prompt for webextensions installed using mozAddonManager
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Prompt users with permissions for third-party webextensions installs
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms
</div></div>
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
====== Logged bugs ======
<div class="mw-collapsible-content">
[ Bug xxxxxxx] - Misaligned icon and webextension name in permissions doorhanger<br/>
</div>
</div>
= Sign off =
== Criteria ==
Check list
* All test cases should be executed
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)
== Results ==
'''Nightly testing'''<br />
List of OSes that will be covered by testing<br />
*Link for the tests run
** Full Test suite, use template from []
'''Merge to Beta Sign-off'''
List of OSes that will be covered by testing<br />
*Link for the tests run
** Full Test suite
== Checklist ==
{| class="wikitable" style="width:60%"
|-
! Exit Criteria !! Status !! Notes/Details
|-
|  Testing Prerequisites (specs, use cases)
| style="text-align:center;" | 
| style="text-align:center;" |
|-
|  Testing Infrastructure setup
|style="text-align:center;" |  ||
|-
|  Test Plan Creation
| style="text-align:center;" | 05-22-2018  ||
|-
|  Test Cases Creation
|style="text-align:center;" |  05-21-2018 || N/A
|-
|  Full Functional Tests Execution
|style="text-align:center;" | 
|style="text-align:center;" | 
|-
|  Automation Coverage ||
|style="text-align:center;" |
|-
|  Performance Testing
|style="text-align:center;" |  ||
|-
|  All Defects Logged || ||
|-
|  Critical/Blockers Fixed and Verified || ||
|-
|  Metrics/Telemetry||
|style="text-align:center;" |
|-
|  QA Signoff  - Nightly Release||
|style="text-align:center;" | Email to be sent
|-
|  QA Beta - Full Testing
|style="text-align:center;" |  ||
|-
|  QA Signoff  - Beta Release ||
|style="text-align:center;" | Email to be sent
|}
CosminB
231

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/1194223"