Confirmed users
3,727
edits
QA/BrowserID/TestPlan: Difference between revisions
no edit summary
No edit summary |
No edit summary |
||
| Line 14: | Line 14: | ||
== Links and Documentation == | == Links and Documentation == | ||
* Identity main site: https://wiki.mozilla.org/Identity | * Identity main site: https://wiki.mozilla.org/Identity | ||
* Production server: https://browserid.org | ** Production server: https://browserid.org | ||
* BrowserID | ** BrowserID Introduction and Demo: https://browserid.org/users | ||
* BrowserID How-To for developers: https://browserid.org/developers.html | ** BrowserID How-To for developers: https://browserid.org/developers.html | ||
* Services QA Sites: | * Services QA Sites: | ||
* Mozilla QA main site: https://wiki.mozilla.org/QA | ** Mozilla QA main site: https://wiki.mozilla.org/QA | ||
* Mozilla QA main site: http://quality.mozilla.org/ | ** Mozilla QA main site: http://quality.mozilla.org/ | ||
* Browser Technologies QA: https://wiki.mozilla.org/QA/Browser_Technologies | ** Browser Technologies QA: https://wiki.mozilla.org/QA/Browser_Technologies | ||
* Services QA: https://wiki.mozilla.org/QA/Browser_Technologies/Services | ** Services QA: https://wiki.mozilla.org/QA/Browser_Technologies/Services | ||
== Weekly Test Schedules == | == Weekly Test Schedules == | ||
| Line 28: | Line 28: | ||
* Thursdays/Fridays: open testing and experimentation by Dev, QA, and community | * Thursdays/Fridays: open testing and experimentation by Dev, QA, and community | ||
* Following week: | * Following week: | ||
** Monday - | ** Monday - Wednesday: QA testing and sign off of current deployment | ||
== Weekly Meetings == | == Weekly Meetings == | ||
| Line 47: | Line 47: | ||
** Product: mozilla.org | ** Product: mozilla.org | ||
** Component: Server Operations: Labs | ** Component: Server Operations: Labs | ||
== BrowserID Client/Server Bugs/Issues == | == BrowserID Client/Server Bugs/Issues == | ||
* Add/update issues: https://github.com/mozilla/browserid/issues | * Add/update issues: https://github.com/mozilla/browserid/issues | ||
= Client/Server Test Environment = | = Client/Server Test Environment = | ||
== Beta test environment == | |||
* TBD | |||
== Prod test environment == | |||
* Server-Side (PIA/IP) | |||
** Acts as the Implementation Provider (IP) until browsers have native support. | |||
** https://browserid.org | |||
* Server Access (Localhost examples): | |||
** https://browserid.org/verify: http://127.0.0.1:10000 | |||
** http://rp.eyedee.me: http://127.0.0.1:10001 | |||
** https://browserid.org: http://127.0.0.1:10002 | |||
** Are there any others? | |||
* Client-Side (RP) | |||
** Acts as the Relying Party (RP) until there are more public clients to test. | |||
** www.myfavoritebeer.org | |||
= Supported OS and Browsers = | = Supported OS and Browsers = | ||
== Operating Systems == | |||
* PC: Win 7, WinXP, Vista (in priority order) | |||
* Mac 10.6, 10.7, 10.5 | |||
* Linux: Fedora 14/15, EL5/6, Ubuntu | |||
== Browsers == | |||
* Minimal support would have to include a native browser plus Firefox | |||
** Windows: IE 8, FF 4/5, Google Chrome latest | |||
** Mac: Safari latest, FF 4/5, Google Chrome latest | |||
** Linux: FF 3/4/5, Google Chrome latest | |||
= Major Test Areas = | |||
== Code Verification == | |||
Verify non-QA-testable fixes/code changes by direct inspection (as required) | |||
* Server Source: https://github.com/mozilla/browserid | |||
* Client Source: https://github.com/lloyd/myfavoritebeer.org/ | |||
== Installation/Deployment Verification == | |||
QA needs access to Beta test environment in order to verify the deployment to server side (PIA/IP) and client side (RP). It would be helpful for QA to verify the weekly deployment to the server, along with any client-side changes (JavaScript library, JavaScript API). For early trains, this should be covered by information given in the BrowserID deployment ticket (Bugzilla). This would amount to a delta from the previous train that QA can record as part of their testing. | |||
For subsequent trains, the Beta test environment or a QA environment (deployed with the same build) could be used to verify server deployments and client-side changes. | |||
= | == Sanity/Acceptance == | ||
== | Manual and automated testing on client-side to pass a minimal level of acceptance without which, QA testing of BrowserID can not proceed: | ||
== | |||
== | == Basic Functional == | ||
== | Manual and automated testing on the client and the server to verify basic functionality of BrowserID: | ||
== UI == | |||
Manual and automated testing on the client and the server to cover all aspects of the current UI: | |||
== Bug Verification == | |||
Manual testing of bugs/issues resolved for this weekly cycle of testing. | |||
Test cases generated during this testing can be moved to an automation tool for bug regression (see below). | |||
== Bug Regression == | |||
Manual and automated testing of Verified/Closed bugs and issues from previous cycles of testing. | |||
Ideally, this would be in the form of a quick automated test/script to check key fixes/changes/updates. | |||
QA to work with Dev and PM to come up with a list of bugs/issues candidates to from the test cases for manual and automated testing. | |||
== OS/Browser integration == | |||
Manual testing/verification of the required OS and browser configurations. | |||
= Areas Not Covered = | = Areas Not Covered = | ||
Currently, QA has not automation configured for the weekly BrowserID testing. | |||
TBD: Client-side automation | |||
TBD: Server-side automation, primarily to cover the API | |||
= Setting Up Test Emails = | = Setting Up Test Emails = | ||
== Email-side (PIA) == | |||
* This is the Primary Identity Authority (PIA) for all the transactions. | |||
* Examples for testing: Mozilla, Yahoo, Google, ISPs | |||
* Also available for testing are sites like Mailinator | |||
== Using Internal Email Accounts == | |||
* If secure enough to do so: <mozilla name>+tag@mozilla.com | |||
* Otherwise: Create emails at www.mailinator.com | |||
* Do we have other tested (re: reliable) methods for creating test/dummy email accounts? | |||
= Open Questions/Issues = | = Open Questions/Issues = | ||