QA/Firefox 3.next/Test Plan:AddonsManagerRedesign: Difference between revisions

From MozillaWiki
< QA
Jump to navigation Jump to search
No edit summary
 
(6 intermediate revisions by 2 users not shown)
Line 26: Line 26:
* [https://bugzilla.mozilla.org/show_bug.cgi?id=461973 Extension Manager API Rewrite]
* [https://bugzilla.mozilla.org/show_bug.cgi?id=461973 Extension Manager API Rewrite]
* [https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&component=Discovery%20Pane&product=addons.mozilla.org Discovery Pane]
* [https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&component=Discovery%20Pane&product=addons.mozilla.org Discovery Pane]
* [http://bugzilla.mozilla.org/show_bug.cgi?id=564870 Add-on Compatibility Reporter]
* [http://bugzilla.mozilla.org/show_bug.cgi?id=565930 Add-on Compatibility Reporter]
* [http://bugzilla.mozilla.org/show_bug.cgi?id=569667 Pull More AMO Data]
* [http://bugzilla.mozilla.org/show_bug.cgi?id=569667 Pull More AMO Data]
* [https://bugzilla.mozilla.org/show_bug.cgi?id=563909 Accessibility]
* [https://bugzilla.mozilla.org/show_bug.cgi?id=563909 Accessibility]
Line 46: Line 46:
|-
|-
| [[#Beta1]]
| [[#Beta1]]
| N/A
| Ready to Ship
|-
|-
| [[#Beta2]]
| [[#Beta2]]
Line 285: Line 285:
|-
|-
| Add-ons Manager (interface and backend):
| Add-ons Manager (interface and backend):
* Blocker: [https://bugzilla.mozilla.org/buglist.cgi?type0-1-0=substring&field0-1-0=cf_blocking_193&field0-0-0=blocked&type0-0-1=substring&field0-0-1=status_whiteboard&query_format=advanced&value0-1-0=alpha5%2B&value0-0-1=AddonsRewrite&type0-0-0=anywordssubstr&value0-0-0=461973%2C550048%2C553169%2C558289%2C558289 alpha5]
* Blocker: [http://tinyurl.com/24858er alpha5]
* Blocker: [https://bugzilla.mozilla.org/buglist.cgi?type0-1-0=substring&field0-1-0=cf_blocking_193&field0-0-0=blocked&type0-0-1=substring&field0-0-1=status_whiteboard&query_format=advanced&value0-1-0=beta1%2B&value0-0-1=AddonsRewrite&type0-0-0=anywordssubstr&value0-0-0=461973%2C550048%2C553169%2C558289%2C558289 beta1]
* Blocker: [http://tinyurl.com/2fjgzl9 beta1]
* Blocker: [https://bugzilla.mozilla.org/buglist.cgi?type0-1-0=substring&field0-1-0=cf_blocking_193&field0-0-0=blocked&type0-0-1=substring&field0-0-1=status_whiteboard&query_format=advanced&value0-1-0=final%2B&value0-0-1=AddonsRewrite&type0-0-0=anywordssubstr&value0-0-0=461973%2C550048%2C553169%2C558289%2C558289 final]
* Blocker: [http://tinyurl.com/2fn33bb beta2]
* Triage: [https://bugzilla.mozilla.org/buglist.cgi?keywords=uiwanted&query_format=advanced&keywords_type=allwords&field0-0-0=blocked&type0-0-0=anywordssubstr&value0-0-0=461973%2C550048%2C553169%2C558289&type0-0-1=substring&value0-0-1=AddonsRewrite&field0-0-1=status_whiteboard uiwanted]
* Blocker: [http://tinyurl.com/2b45cr7 final]
* Triage: [https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&field0-0-0=blocked&type0-0-0=anywordssubstr&value0-0-0=461973%2C550048%2C553169%2C558289%2C558289&type0-0-1=substring&value0-0-1=AddonsRewrite&field0-0-1=status_whiteboard all]
* Triage: [http://tinyurl.com/235oa2m uiwanted]
* Triage: [https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&field0-0-0=blocked&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&type0-0-1=substring&value0-0-1=AddonsRewrite&type0-0-0=anywordssubstr&field0-0-1=status_whiteboard&value0-0-0=461973%2C550048%2C553169%2C558289%2C558289 open]
* Triage: [http://tinyurl.com/2blqg7h all]
* Triage: [https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&field0-0-0=blocked&bug_status=UNCONFIRMED&type0-0-0=anywordssubstr&value0-0-0=461973%2C550048%2C553169%2C558289&type0-0-1=substring&value0-0-1=AddonsRewrite&field0-0-1=status_whiteboard unconfirmed]
* Triage: [http://tinyurl.com/242j28v open]
* Tests: [https://bugzilla.mozilla.org/buglist.cgi?type0-1-0=notsubstring&field0-1-0=flagtypes.name&field0-0-0=blocked&type0-0-1=substring&type0-1-1=notsubstring&field0-0-1=status_whiteboard&resolution=FIXED&resolution=---&query_format=advanced&value0-1-0=in-testsuite&value0-1-1=in-litmus&field0-1-1=flagtypes.name&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=RESOLVED&bug_status=VERIFIED&value0-0-1=AddonsRewrite&type0-0-0=anywordssubstr&value0-0-0=461973%2C550048%2C553169%2C558289%20in-testsuite%3F flags missing]
* Triage: [http://tinyurl.com/2496tsm unconfirmed]
* Tests: [https://bugzilla.mozilla.org/buglist.cgi?type0-1-0=substring&field0-1-0=flagtypes.name&field0-0-0=blocked&type0-0-1=substring&field0-0-1=status_whiteboard&resolution=FIXED&resolution=---&query_format=advanced&value0-1-0=in-testsuite%3F&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=RESOLVED&bug_status=VERIFIED&value0-0-1=AddonsRewrite&type0-0-0=anywordssubstr&value0-0-0=461973%2C550048%2C553169%2C558289%20in-testsuite%3F in-testsuite?]
* Tests: [http://tinyurl.com/2fgn8y3 flags missing]
* Tests: [https://bugzilla.mozilla.org/buglist.cgi?type0-1-0=substring&field0-1-0=flagtypes.name&field0-0-0=blocked&type0-0-1=substring&field0-0-1=status_whiteboard&resolution=FIXED&resolution=---&query_format=advanced&value0-1-0=in-testsuite%2B&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=RESOLVED&bug_status=VERIFIED&value0-0-1=AddonsRewrite&type0-0-0=anywordssubstr&value0-0-0=461973%2C550048%2C553169%2C558289%20in-testsuite%3F in-testsuite+]
* Tests: [http://tinyurl.com/28wrv8o in-testsuite?]
* Tests: [https://bugzilla.mozilla.org/buglist.cgi?type0-1-0=substring&field0-1-0=flagtypes.name&field0-0-0=blocked&type0-0-1=substring&field0-0-1=status_whiteboard&resolution=FIXED&resolution=---&query_format=advanced&value0-1-0=in-litmus%3F&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=RESOLVED&bug_status=VERIFIED&value0-0-1=AddonsRewrite&type0-0-0=anywordssubstr&value0-0-0=461973%2C550048%2C553169%2C558289%20in-testsuite%3F in-litmus?]
* Tests: [http://tinyurl.com/2843lm8 in-testsuite+]
* Tests: [https://bugzilla.mozilla.org/buglist.cgi?type0-1-0=substring&field0-1-0=flagtypes.name&field0-0-0=blocked&type0-0-1=substring&field0-0-1=status_whiteboard&resolution=FIXED&resolution=---&query_format=advanced&value0-1-0=in-litmus%2B&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=RESOLVED&bug_status=VERIFIED&value0-0-1=AddonsRewrite&type0-0-0=anywordssubstr&value0-0-0=461973%2C550048%2C553169%2C558289%20in-testsuite%3F in-litmus+]
* Tests: [http://tinyurl.com/27acxzj in-litmus?]
|<br>
* Tests: [http://tinyurl.com/29jh5b5 in-litmus+]
* Bugs which need feedback from the UX team
|
* All Bugs which have been filed for that new feature
* Bugs which haven't been fixed yet
* Bugs which haven't been confirmed yet
* Blocker bugs for Alpha5
* Blocker bugs for Beta1
* Blocker bugs for Final
* Bugs with missing flags
* Bugs which need automated tests
* Bugs which have automated tests
* Bugs which need manual tests
* Bugs which have manual tests
|-
|-
| Discovery Pane:
| Discovery Pane:
Line 315: Line 304:
|-
|-
| Add-ons Compatibility Reporter:
| Add-ons Compatibility Reporter:
* Triage: [http://tinyurl.com/2ftkw4u all]
* Triage: [http://tinyurl.com/28bvk7l open]
* Triage: [http://tinyurl.com/2ap8hgm unconfirmed]
|
|
|-
|-
Line 321: Line 313:
|-
|-
| Top Issues:
| Top Issues:
* {{bug|553455}}
* {{bug|565930}} - ACR inclusion?
* {{bug|553493}}
* {{bug|520124}} - Personas on top of custom themes
* {{bug|562622}}
* {{bug|553455}} - Missing notification box
* {{bug|562902}}
* {{bug|553493}} - UI for restartless add-ons
* {{bug|565610}}
* {{bug|562622}} - Update design mockups
* {{bug|520124}}
* {{bug|562902}} - Detail pane mockups
| Open bugs which blocks testing of a specific area.
| Open bugs which blocks testing of a specific area.
* No notification of addons being installed
* Complete ui for installing restart-less addons
* Update design mockups
* Detail View design mockups
* Can not overlay about: urls
* Personas on top of other custom themes
|}
|}
    
    
Line 358: Line 344:


== Accessibility ==   
== Accessibility ==   
* Details of feature accessibility test requirements
* Keyboard accessibility: Test that all actions can be performed through the keyboard, without the use of a mouse. Items can either be interacted with directly (buttons to perform actions, for example), or be accessed through a popup/context menu of any given item. Success is met when all interactive items can be tabbed to and all features are reachable through either an interactive control or the context menu of an item.
* Themeing testing: Is the UI still fully readable if the operating system's theme is changed to high-contrast, inverted colors etc? Success is met when all controls are still readable in interactive and noninteractive states when changing the operating system's color scheme.
* Screen reader exposure: Are all controls and items exposed with all available information to screen readers? Success is met when all controls speak with their name and control type (like button etc.).


[[#Feature_Test_Items|top]]
[[#Feature_Test_Items|top]]

Latest revision as of 21:47, 30 June 2010

« Firefox 4 TestPlan

Add-ons Manager Redesign

Area Details
Development Status: In progress (10/06/09)
Feature Testing: In progress (10/06/09)
Team: By group:
  • Jennifer Boriss (UX)
  • Dave Townsend, Blair McBride, Robert Strong, Ben Parr, Benedict Hsieh (Add-ons Manager)
  • Justin Scott, Jeff Balogh (Discovery Pane)
  • Brian King, David McNamara (Add-on Compatibility Reporter)
  • Henrik Skupin, Marco Zehe, Raymond Etornam (QA)
Tracking Bugs By sub-project:


Feature Description

This redesign will substantially improve the basic functionality of the a add-ons manager, such as installation and configuration of add-ons, as well as incorporate additional features and functionality, such as use of Personas and Jetpacks. The design will also remove parts of the manager that interrupt and distract user tasks.

Feature Release Readiness Assessment

The table below provides a top level go/no go assessment of whether the feature is release ready for the given milestone. Each milestone link references a section below that discusses the criteria and evaluation that went into the QA go/no-go decision.

Milestone Assessment
#Beta1 Ready to Ship
#Beta2 N/A
#Beta3 N/A

top

Feature Documentation

Item Description Status
#Project_Wiki Wiki Links to all feature related entries
#Developer_Links (blogs) Developer links to feature related sites
#Other_Docs Web links to feature related sites
#Developer_QA_Review Details from developer and qa discussions regarding feature test strategies and issues.

top

Feature Bug Management

Item Description Status
#Bug_Tracking Top level bugs tracking feature
#Bug_Verification Feature bugs that need verification
#Bug_Triage Links triage bug tasks

top

Feature Test Items

The table below provides a breakdown of all feature items that should be covered and how they will be tested. Not all items will be covered by internal QA team members. It is important to list what should be covered. If it is not covered, list it as not covered.

Note: not all items listed below will apply for a given feature

Test Item Description Covered By Status
Item 1 Item 1 Description Developer Tests
Item 2 Item 2 Description Beta tester exposure
#Localization Feature localization
#Accessibility Feature accessibility
#Plugins Plugins compatibility
#Addons Addons compatibility
#Topsites Top internet sites compatibilities

top


Feature Tests

Automated Tests

Item Description Status
#Developer_Tests Links to automated developer tests
#Mozmill_Tests Links to automated mozmill feature test cases

Manual Tests

Item Description Status
#Smoke_Tests link to smoke tests
#Regression_Tests link to BFT and/or regression tests
#Functional_Tests link to FFT and/or complete functional tests

top

Community Test Events

Item Description Status
#Testdays Links to test day event results for feature
#Bugdays Links to bug day event results for feature
#Meetups Links to Meetup events for feature

top

Feature Documentation Details

Project Wiki

top

Developer Links

top

Other Docs

top

Developer QA Review

The QA person responsible for the feature should hold a formal interview with the lead developer or feature champion. Below are questions that should be asked in the interview:

  • Do we have automated tests for the feature?
  • What do they cover?
  • What do they not cover?
  • How well do they cover the feature?
  • What are the important areas we should focus on?
  • What are the dependencies?
  • What is our comfort level with this feature in its current state?
  • What feedback would you like from QA?

top

Feature Release Readiness Assessment Details

Beta1

Beta2

Beta3

Feature Bug Management Details

Bug Tracking

  • Top level bugs tracking feature. Include any relevant bug queries that are helpful for tracking feature status.
Query Name Description
Add-ons Manager (interface and backend):
Discovery Pane:
Add-ons Compatibility Reporter:
Pull More AMO Data into the Add-ons Manager:
Top Issues: Open bugs which blocks testing of a specific area.

top

Bug Verification

top

Bug Triage

  • Bug triage information

top

Feature Test Items Details

Localization

  • Details of feature localization test requirements

top

Accessibility

  • Keyboard accessibility: Test that all actions can be performed through the keyboard, without the use of a mouse. Items can either be interacted with directly (buttons to perform actions, for example), or be accessed through a popup/context menu of any given item. Success is met when all interactive items can be tabbed to and all features are reachable through either an interactive control or the context menu of an item.
  • Themeing testing: Is the UI still fully readable if the operating system's theme is changed to high-contrast, inverted colors etc? Success is met when all controls are still readable in interactive and noninteractive states when changing the operating system's color scheme.
  • Screen reader exposure: Are all controls and items exposed with all available information to screen readers? Success is met when all controls speak with their name and control type (like button etc.).

top

Plugins

  • Details of plugins compatibility test requirements

top

Addons

  • Details of addons compatibility

top

Topsites

  • Details of top internet sites test requirements

top

Feature Tests Details

To test all of the new features which will come with Firefox 4 we have setup a test plan which can be found on Google Spreadsheets.

Automated Tests Details

Developer Tests

  • Links to automated developer tests

top

Mozmill Tests

If a particular feature needs manual tests which should also be covered by Mozmill tests please add the "[mozmill-test-needed]" whiteboard entry to the feature implementation or regression bug.

List of Mozmill Tests:

  • Links to automated mozmill feature test cases

top

Manual Tests Details

Areas which cannot be tested by our automated test harnesses will need a good set of manual tests on Litmus. We cover those tests in a Google Spreadsheet.

Smoke_Tests

  • links to litmus smoke tests or description

top

Regression_Tests

  • links to litmus BFT and/or regression tests description

top

Functional_Tests

  • links to litmus FFT and/or complete functional tests description

top

Community Test Events Details

Testdays

Bugdays

None

Meetups

None

Retrieved from "https://wiki.allizom.org/index.php?title=QA/Firefox_3.next/Test_Plan:AddonsManagerRedesign&oldid=235545"