QA/Test Pilot Test Plan: Difference between revisions

From MozillaWiki
< QA
Jump to navigation Jump to search
mNo edit summary
 
(19 intermediate revisions by 2 users not shown)
Line 1: Line 1:
;Test Pilot Test Plan
;Test Pilot Test Plan
''last update by tracy (1-26-2010 - This is a work in progress)''


= Overview =
= Overview =
Test Pilot is a platform aimed at collecting structured user feedback from Firefox and other Mozilla Labs experiments.
Test Pilot is a platform aimed at collecting structured user feedback and log data from Firefox and other Mozilla Labs experiments.


Testing Weave will present some unique challenges in that data from many areas of the browser are to be stored and synced securely across multiple browser versions and multiple operating systems, including mobile.
[https://wiki.mozilla.org/Labs/Test_Pilot Mozilla Labs - Test Pilot Project Page]<br>
 
[https://testpilot.mozillalabs.com/ Test Pilot Home Page]<br>
[https://wiki.mozilla.org/Labs/Test_Pilot| Mozilla Labs - Test Pilot Project Page]<br>
[http://mozillalabs.com/testpilot/ Test Pilot Blog]<br>
[https://testpilot.mozillalabs.com/ Test Pilot Home Page]
[http://spreadsheets.google.com/ccc?key=0AhClKN8axlADdHhScUNTbC0xMlF1TXpNeWYzNTZIX3c&hl=en Test Pilot Weekly Planner]
[http://mozillalabs.com/testpilot/ Test Pilot Blog]


== Builds ==
== Builds ==
Line 20: Line 17:


* Areas of test coverage
* Areas of test coverage
* Installing/Upgrading the extension
** Installing/Upgrading the extension
* General User Interface
** Extension UI points
** Access to various pages
*** Menu Items
** Verify interfaces for surveys, experiments, etc.
**** Notification Settings...
*** Verify data is collected as expected
**** All Your Settings (and it's xul dialog)
*** Confirm data is uploaded with feedback as to its status
**** About Test Pilot
** Verify interfaces for study access.
*** studies are present, active and submittable (when appropriate)
*** multiple active studies.
*** Use instructions at [https://wiki.mozilla.org/Labs/Test_Pilot/debug Test Pilot Debug]
** Access to various pages linked from the extension or survey
 
* Verify bug P1/P2 bug fixes a couple times a month


* Unit tests from developers?
* Unit tests from developers?
Line 39: Line 43:
* Operating systems covered
* Operating systems covered
** Windows
** Windows
*** '''XP (primary)'''
*** '''7 (primary)'''
*** 7 (spot check if time allows)
*** XP (spot check if time allows)
** Apple
** Apple
*** '''Snow Leopard (10.6) (primary)'''
*** '''Snow Leopard (10.6) (primary)'''
Line 50: Line 54:


= Bugs =
= Bugs =
* [https://bugzilla.mozilla.org/buglist.cgi?classification=Other;query_format=advanced;bug_status=UNCONFIRMED;bug_status=NEW;bug_status=ASSIGNED;bug_status=REOPENED;component=Test%20Pilot;product=Mozilla%20 All open Test Pilot bugs]
* [https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&component=Test%20Pilot&product=Mozilla%20Labs&classification=Other All open Test Pilot bugs]
* [https://bugzilla.mozilla.org/buglist.cgi?resolution=FIXED;classification=Other;chfieldto=Now;query_format=advanced;chfieldfrom=-1m;bug_status=RESOLVED;component=Test%20Pilot;product=Mozilla%20Labs Bugs fixed in the last month (need VERIFICATION)]
* [https://bugzilla.mozilla.org/buglist.cgi?resolution=FIXED;classification=Other;chfieldto=Now;query_format=advanced;chfieldfrom=-1m;bug_status=RESOLVED;component=Test%20Pilot;product=Mozilla%20Labs Bugs fixed in the last month (need VERIFICATION)]


= Contacts=
= Contacts=
Developer - Jono
* Developer - Jono
QA - Tracy
* QA - Tracy

Latest revision as of 22:09, 19 April 2010

Test Pilot Test Plan

Overview

Test Pilot is a platform aimed at collecting structured user feedback and log data from Firefox and other Mozilla Labs experiments.

Mozilla Labs - Test Pilot Project Page
Test Pilot Home Page
Test Pilot Blog
Test Pilot Weekly Planner

Builds

Test Strategy

Mozilla QA will be taking a lightweight testing approach to Test Pilot.

  • Areas of test coverage
    • Installing/Upgrading the extension
    • Extension UI points
      • Menu Items
        • Notification Settings...
        • All Your Settings (and it's xul dialog)
        • About Test Pilot
    • Verify interfaces for study access.
      • studies are present, active and submittable (when appropriate)
      • multiple active studies.
      • Use instructions at Test Pilot Debug
    • Access to various pages linked from the extension or survey
  • Verify bug P1/P2 bug fixes a couple times a month
  • Unit tests from developers?
  • Areas not tested
    • data integrity/correctness on server end
    • long term regular interaction with Test Pilot tests (this is better left for our community)
  • Browser Versions covered
    • Firefox 3.6 (primary)
    • Firefox next (spot check if time allows)
  • Operating systems covered
    • Windows
      • 7 (primary)
      • XP (spot check if time allows)
    • Apple
      • Snow Leopard (10.6) (primary)
    • Linux (spot check if time allows)

Schedule Scoping

  • We would like QA to give each new version of the Test Pilot Extension itself as well a each new test a quick once over before it is released as a sanity check. We'll be flexible in doing this smoke testing any time in the development cycle for that release.

Bugs

Contacts

  • Developer - Jono
  • QA - Tracy
Retrieved from "https://wiki.allizom.org/index.php?title=QA/Test_Pilot_Test_Plan&oldid=216866"