Add-ons/QA/Testplan/NewAMOMobile: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(Created page with "= Overview = The new add-on submission flow for listed and unlisted add-ons, uploading a new version or a new file, for the same version, together with add-ons with mixed vers...")
 
 
(22 intermediate revisions by 2 users not shown)
Line 1: Line 1:
= Overview =
= Overview =
The new add-on submission flow for listed and unlisted add-ons, uploading a new version or a new file, for the same version, together with add-ons with mixed versions, came in a moment when add-on developers needed clarification towards an easy way to finalize the add-on submission and distribution process, and also have multiple distribution channels for their add-ons without any weird workarounds.  
New AMO Mobile design(applied for Desktop Mobile view too) with a modern and simplified look, oriented to users and ease of utilization.  


== Purpose ==
== Purpose ==
Simplify the submission flow by reducing the number of steps required to submit an add-on.
A more attractive AMO page with a more easy and intuitive way to install and search for Add-ons and Themes
Have mixed versions (multiple distribution) for the same add-on.


=== Entry Criteria ===
=== Entry Criteria ===
* QA has access to PRD and some mocks (found in bugs)
* QA has access to [https://mozilla.invisionapp.com/share/GY8O3Z4ZE#/screens mocks] . Any other info were found in bugs logged in GitHub
* The feature has landed in production: https://addons.mozilla.org/en-US/firefox/  
* The feature has landed in -dev : https://addons-dev.allizom.org/en-US/firefox/.
Mobile release is set for March 9th.
 
=== Current status ===
* The feature is partially in -dev . Still under development


=== Exit Criteria ===
=== Exit Criteria ===
Line 18: Line 21:
== Scope ==
== Scope ==
===what's in scope?===
===what's in scope?===
* Add-on distribution
* AMO mobile appearance
* Listed add-on submission flow
* Behavior of AMO mobile in logged in \ logged out states
* Unlisted add-on submission flow
* Install\Uninstall of Add-ons and Themes
* Add-on with mixed versions
* Search for Add-ons and Themes. Sorting the search results. Search suggestions.
* Add-on update submission flow
* Add-on and Themes reviews and ratings
* New file (inside of a version) submission flow
* Add-ons and Themes categories
* Localized add-ons
* AMO mobile pages in different languages and RTL view
* Versions availability in AMO public pages
* Collections - not sure if this will be implemented but is present on mocks
* Listing visibility - Visible/Invisible
* Versions availability in AMO mobile public pages
* Themes preview
* User settings
* Page errors


===what's out of scope?===
===what's out of scope?===
* Add-ons functionality
* Add-ons functionality
* Translations in different languages . This will not exclude testing on another languages for UI issues related to different length of strings


== Ownership ==
== Ownership ==
Product Manager: [mailto:jorge@mozilla.com Jorge Villalobos]; irc nick :jorgev<br />
Product Manager: [mailto:mmacpherson@mozilla.com Matthew Riley MacPherson]; irc nick :tofumatt<br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br />  
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br />  
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br />
Add-ons QA: [mailto:valentina.peleskei@softvision.ro Valentina Virlics]; irc nick :ValentinaV<br />
Add-ons QA: [mailto:rares.macarie@softvision.ro Rares Macarie]; irc nick :RaresM<br />
Add-ons QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br />


= Requirements for testing =
= Requirements for testing =
== Environments ==
== Environments ==
* Windows
* Android phone, tablets
* Mac OS


== Servers ==
== Servers ==
Line 48: Line 55:


== Channels ==
== Channels ==
* Release
* [http://archive.mozilla.org/pub/mobile/releases/51.0/android-api-15/en-US/fennec-51.0.en-US.android-arm.apk Release]
* Nightly
* [http://archive.mozilla.org/pub/mobile/nightly/2017/01/2017-01-30-11-03-29-mozilla-central-android-api-15/fennec-54.0a1.multi.android-arm.apk Nightly]
* Unbranded builds
* [http://archive.mozilla.org/pub/mobile/tinderbox-builds/mozilla-release-android-api-15/1485542540/fennec-51.0.1.en-US.android-arm.apk Unbranded builds]


= Test Strategy =  
= Test Strategy =  
Line 60: Line 67:
|-
|-
| Start project  
| Start project  
|style="text-align:center;" | 31.08.2016 ||  
|style="text-align:center;" | 11.01.2016 ||  
|-
|-
| Study PRD/mocks received  
| Study PRD/mocks received  
|style="text-align:center;" | 30.09.2016 ||  
|style="text-align:center;" | 11.01.2017 ||  
|-
|-
| QA - Test plan creation  
| QA - Test plan creation  
|style="text-align:center;" | 03.01.2017 || 06.01.2017
|style="text-align:center;" | 30.01.2017 || 31.01.2017
|-
|-
| QA - Test cases preparation  
| QA - Test cases preparation  
|style="text-align:center;" | 10.01.2017 || 11.01.2017  
|style="text-align:center;" | 30.01.2017 || 01.02.2017  
|-
|-
| QA - Test cases execution   
| QA - Test cases execution   
|style="text-align:center;" | 11.01.2017 || 18.01.2017  
|style="text-align:center;" | 01.02.2017 || 08.03.2017  
|-
|-
| Release Date  
| Release Date  
|style="text-align:center;" | 26.01.2017 ||  
|style="text-align:center;" | 09.03.2017 ||  
|}
|}


Line 85: Line 92:
| Test plan creation || Mozilla wiki
| Test plan creation || Mozilla wiki
|-
|-
| Test case creation || [https://testrail.stage.mozaws.net/index.php?/suites/view/99 TestRail] / [https://docs.google.com/document/d/1_U9GIa_q1GVAOYviDvE44M4dLDp329WxGeDViwmdCPI/edit# Google docs]  
| Test case creation || [TBD TestRail] / [https://docs.google.com/spreadsheets/d/1-dOF7KIgbRqSXtVFix_xOBT0g4JB6Vst1tXGTqSoFOo/edit#gid=0 Google docs]  
|-
|-
| Test case execution || [https://testrail.stage.mozaws.net/index.php?/suites/view/99 TestRail]
| Test case execution || [TBD TestRail]
|-
|-
| Bugs management || Github
| Bugs management || Github
Line 93: Line 100:


= References =
= References =
  * PRD - https://docs.google.com/document/d/1v32JqoLcn_6MfviFXOY0UTq-tigifTllL3yOUyn0-1M/edit
  * PRD - https://mozilla.invisionapp.com/share/GY8O3Z4ZE#/screens
* Tracker: [https://github.com/mozilla/addons-server/issues/3465 3465]


= Testcases =  
= Testcases =  
== Test Areas ==
== Test Areas ==
* Submission of add-ons and webextensions
* AMO Mobile


== Test suite ==
== Test suite ==
  Full Test suite - [https://testrail.stage.mozaws.net/index.php?/suites/view/99 Link]
  Gdoc Test suite - [https://docs.google.com/spreadsheets/d/1-dOF7KIgbRqSXtVFix_xOBT0g4JB6Vst1tXGTqSoFOo/edit#gid=0 Link]
TestRail - TBD


= Bug Work =
= Bug Work =
* Tracking bug - [https://github.com/mozilla/addons-server/issues/3465 3465]   
* All Feature bugs:
* All Feature bugs:
** [https://github.com/mozilla/addons-server/labels/version%20channel Open]   
** [https://github.com/mozilla/addons-frontend/labels/mobile%20pages%20v1 Open V1]   
** [https://github.com/mozilla/addons-server/issues?q=label%3A%22version+channel%22+is%3Aclosed Closed]
** [https://github.com/mozilla/addons-frontend/issues?q=label%3A%22mobile+pages+v1%22+is%3Aclosed Closed V1]
** [https://github.com/mozilla/addons-frontend/labels/mobile%20pages%20v2 Open V2]
** [https://github.com/mozilla/addons-frontend/issues?q=label%3A%22mobile+pages+v2%22+is%3Aclosed Closed V2]


<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
Line 113: Line 121:
====== Bug fix verification ======
====== Bug fix verification ======
<div class="mw-collapsible-content">
<div class="mw-collapsible-content">
* [https://github.com/mozilla/addons-server/issues/4081 4081] - Last step of new file submission mentions "version" submission
* [https://github.com/mozilla/addons-frontend/issues?q=label%3A%22mobile+pages+v1%22+label%3A%22verified+fixed%22+is%3Aclosed Verified Fixed V1]  
* [https://github.com/mozilla/addons-server/issues/4134 4134] - "Lastest version" or "Latest version" ?
* [https://github.com/mozilla/addons-server/issues/3704 3704] - Hide unreviewed versions (if there is at least one reviewed version) on the 'all versions' page
* [https://github.com/mozilla/addons-server/issues/4302 4302] - Misleading info while adding a second beta-listed version for an unlisted add-on
* [https://github.com/mozilla/addons-server/issues/4269 4269] - "Incomplete" status for unlisted add-ons in Dev Hub homepage
* [https://github.com/mozilla/addons-server/issues/4395 4395] - Direct links to beta versions are broken
* [https://github.com/mozilla/addons-server/issues/4282 4282] - Click on incomplete status redirects to final step of submission
* [https://github.com/mozilla/addons-server/issues/4245 4245] - Add-on admin page should point to review-unlisted pages when relevant
* [https://github.com/mozilla/addons-server/issues/4024 4024] - Setting an add-on waiting for approval to invisible leaves it with incorrect add-on status
* [https://github.com/mozilla/addons/issues/318 318] - Old version addon links broken, only static link to latest available 
* [https://github.com/mozilla/addons-server/issues/4394 4394] - "Completed" add-on is displayed as incomplete 
* [https://github.com/mozilla/addons-server/issues/4282 4282] - Click on incomplete status redirects to final step of submission 
* [https://github.com/mozilla/addons-server/issues/4420 4420] - Blocking submission for second listed version in 6 steps 
* [https://github.com/mozilla/addons-server/issues/4245 4245] - Add-on admin page should point to review-unlisted pages when relevant 
* [https://github.com/mozilla/addons-server/issues/4398 4398] - the (AMO) and (Self) tags aren't localised 
* [https://github.com/mozilla/addons-server/issues/4022 4022] - Error displayed while adding a developer reply for second unlisted version of an add-on 
* [https://github.com/mozilla/addons-server/issues/4479 4479] - Add-on unlisted review page is returning 500 error for add-ons with mixed unlisted/listed versions 
* [https://github.com/mozilla/addons-server/issues/4484 4484] - Search in "All Unlisted Add-ons" is not working for add-on with mixed versions if first version is unlisted 


</div></div>
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
====== Logged bugs ======
<div class="mw-collapsible-content">
* [https://github.com/mozilla/addons-server/issues/4217 4217] - Beta-channel tooltip from beta-version submission step is not aligned
* [https://github.com/mozilla/addons-server/issues/4220 4220] - Different format for beta versions used in Add-ons and WebExtensions
* [https://github.com/mozilla/addons-server/issues/4226 4226] - "Review History" links for beta versions are useless
* [https://github.com/mozilla/addons-server/issues/4269 4269] - "Incomplete" status for unlisted add-ons in Dev Hub homepage
* [https://github.com/mozilla/addons-server/issues/4282 4282] - Click on incomplete status redirects to final step of submission
* [https://github.com/mozilla/addons-server/issues/4488 4488] - Handle duplicate version better
* [https://github.com/mozilla/addons-server/issues/4491 4491] - No source-code file available anywhere if added during new file submission
* [https://github.com/mozilla/addons-server/issues/4394 4394] - "Completed" add-on is displayed as incomplete 
* [https://github.com/mozilla/addons-server/issues/4302 4302] - Misleading info while adding a second beta-listed version for an unlisted add-on 
* [https://github.com/mozilla/addons-server/issues/4420 4420] - Blocking submission for second listed version in 6 steps 
* [https://github.com/mozilla/addons-server/issues/4427 4427] - Beta add-on asking for resume after deleting first listed version 
* [https://github.com/mozilla/addons-server/issues/4479 4479] - 500 error page is loaded while unlisted version number is clicked from Admin Manage page 
* [https://github.com/mozilla/addons-server/issues/4484 4484] - Search in "All Unlisted Add-ons" is not working for add-on with mixed versions if first version is unlisted 
</div></div>
</div></div>


Latest revision as of 13:26, 6 July 2017

Overview

New AMO Mobile design(applied for Desktop Mobile view too) with a modern and simplified look, oriented to users and ease of utilization.

Purpose

A more attractive AMO page with a more easy and intuitive way to install and search for Add-ons and Themes

Entry Criteria

Mobile release is set for March 9th.

Current status

  • The feature is partially in -dev . Still under development

Exit Criteria

  • All related bugs triaged
  • All blockers fixed
  • All resolved bugs verified by QA
  • Found-fixed bugs rate going down in time

Scope

what's in scope?

  • AMO mobile appearance
  • Behavior of AMO mobile in logged in \ logged out states
  • Install\Uninstall of Add-ons and Themes
  • Search for Add-ons and Themes. Sorting the search results. Search suggestions.
  • Add-on and Themes reviews and ratings
  • Add-ons and Themes categories
  • AMO mobile pages in different languages and RTL view
  • Collections - not sure if this will be implemented but is present on mocks
  • Versions availability in AMO mobile public pages
  • Themes preview
  • User settings
  • Page errors

what's out of scope?

  • Add-ons functionality
  • Translations in different languages . This will not exclude testing on another languages for UI issues related to different length of strings

Ownership

Product Manager: Matthew Riley MacPherson; irc nick :tofumatt
QA Manager: Krupa Raj; irc nick :krupa
QA Lead: Victor Carciu; irc nick :victorc
Add-ons QA: Rares Macarie; irc nick :RaresM
Add-ons QA: Cosmin Badescu; irc nick :CosminB

Requirements for testing

Environments

  • Android phone, tablets

Servers

Channels

Test Strategy

Test Execution Schedule

The following table identifies the anticipated testing period available for test execution.

Project phase Start Date End Date
Start project 11.01.2016
Study PRD/mocks received 11.01.2017
QA - Test plan creation 30.01.2017 31.01.2017
QA - Test cases preparation 30.01.2017 01.02.2017
QA - Test cases execution 01.02.2017 08.03.2017
Release Date 09.03.2017

Testing Tools

Process Tool
Test plan creation Mozilla wiki
Test case creation [TBD TestRail] / Google docs
Test case execution [TBD TestRail]
Bugs management Github

References

* PRD - https://mozilla.invisionapp.com/share/GY8O3Z4ZE#/screens

Testcases

Test Areas

  • AMO Mobile

Test suite

Gdoc Test suite - Link
TestRail - TBD

Bug Work

Bug fix verification

Sign off

Criteria

Check list

  • All test cases should be executed
  • All blockers must be fixed and verified or have an agreed-upon timeline for being fixed
Retrieved from "https://wiki.allizom.org/index.php?title=Add-ons/QA/Testplan/NewAMOMobile&oldid=1175147"