Confirmed users
3,727
edits
QA/BrowserID/TestPlan: Difference between revisions
no edit summary
No edit summary |
No edit summary |
||
| Line 89: | Line 89: | ||
For subsequent trains, the Beta test environment or a QA environment (deployed with the same build) could be used to verify server deployments and client-side changes. | For subsequent trains, the Beta test environment or a QA environment (deployed with the same build) could be used to verify server deployments and client-side changes. | ||
== Sanity/Acceptance == | == Sanity/Acceptance/Smoke == | ||
Manual and automated testing on client-side to pass a minimal level of acceptance without which, QA testing of BrowserID can not proceed: | Small, repeatable set of tests with known, good, expected results. Manual and automated testing on client-side to pass a minimal level of acceptance without which, QA testing of BrowserID can not proceed. This will probably be a very small subset of the basic functional tests or some automated smoke test: TBD | ||
== Bug Verification == | |||
Manual testing of bugs/issues resolved for this weekly cycle of testing. | |||
Test cases generated during this testing can be moved to an automation tool for bug regression (see below). | |||
== Basic Functional == | == Basic Functional == | ||
Manual and automated testing on the client and the server to verify basic functionality of BrowserID: | Manual and automated testing on the client and the server to verify basic functionality of BrowserID: | ||
* Creating an account first | |||
* Creating an account inline (at first use) | |||
* Creating multiple accounts | |||
* Deleting one or more accounts | |||
* Adding additional emails | |||
* Deleting one or more emails (maintaining the account though) | |||
* Forgotten account information - mail, passwords | |||
* Leaving/returning to sites | |||
* Browser restart after creation of identity | |||
== UI == | == UI == | ||
Manual and automated testing on the client and the server to cover all aspects of the current UI: | Manual and automated testing on the client and the server to cover all aspects of the current UI: | ||
* HTML pop-ups | |||
* Account creation UI | |||
* Sign-in UI | |||
* Add a new email address | |||
* This is not me | |||
* Is there anything available inside FF desktop or mobile yet? | |||
* Stay logged in vs. always log out | |||
== Bug Regression == | == Bug Regression == | ||
| Line 108: | Line 124: | ||
== OS/Browser integration == | == OS/Browser integration == | ||
Manual testing/verification of the required OS and browser configurations. | Manual testing/verification of the required OS and browser configurations: | ||
* Verify access and use across browsers on same OS, different OS | |||
* Verify access and use across OS platforms, same browser, different browser | |||
* This includes account creation/update/deletion and email addition/deletion | |||
* OS/Browser-specific local storage verification | |||
* Browser preferences, esp for privacy/security | |||
* Browser synchronization - same platform, across platforms | |||
== Server-side Testing == | |||
* Verify some UI elements and actions only found on the server: | |||
** Manage Accounts page: https://www.browserid.org/manage | |||
* Log inspection to verify accurate information | |||
** Verify content of web server access logs | |||
** Verify content of web server error logs | |||
** Verify content of BrowserID logs | |||
* What other server-specific tests are required? | |||
== Ad Hoc == | |||
* General ease of use | |||
* Email and Password character compatibility | |||
* Valid vs. invalid email formats | |||
* Different accounts using same email/password combos | |||
= Areas Not Covered = | = Areas Not Covered or In Development = | ||
== Server and Client Automation == | |||
Currently, QA has not automation configured for the weekly BrowserID testing. | Currently, QA has not automation configured for the weekly BrowserID testing. | ||
TBD: Client-side automation | TBD: Client-side automation | ||
TBD: Server-side automation, primarily to cover the API | TBD: Server-side automation, primarily to cover the API | ||
== Document Signing == | |||
Not much information here, could use help from PM and Dev if specific QA work is needed. | |||
* https://wiki.mozilla.org/Identity/Doc-signing_use-cases | |||
== Security: VEP, VES, VEC == | |||
QA could use some help from Dev and PM on testable use cases. | |||
Valid/invalid public key | |||
Security/Crypto checks | |||
REF: | |||
https://wiki.mozilla.org/Identity/Verified_Email_Protocol (is this the more current link?) | |||
https://wiki.mozilla.org/Labs/Identity/VerifiedEmailProtocol | |||
== FLOWS/END-TO-END == | |||
Verifying each step in the various flows and also the complete End-To-End flow. This testing will need to be done manually, if possible, for a few weekly trains to get the required tests and tools scoped out. Then, this can be moved over to automated E2E testing (alongside the automated smoke testing). This seems like a better fit for server-side automation, where the appropriate actions and verifications could be tested per flow. | |||
Important flows to test/validate: | |||
* Certificate Provisioning: 8 steps to verify full certificate provisioning path | |||
* Assertion Generation: 4 steps to verify assertion generation path | |||
* Assertion Verification: 6 steps to verify assertion verification path | |||
QA could use some help from PM and Dev here to better define use cases for each step of these flows. Provide ideas/cases for testing each of the steps, plus the included assumptions and requirements. | |||
If applicable we should design both client-side and server-side tests. | |||
QA could also leverage off the design/purpose/coverage of the unit tests. | |||
REF: http://lloyd.io/how-browserid-works | |||
== FireFox Add-On == | |||
For Firefox users with the addon, the addon flow needs tested. | |||
Test design: TBD | |||
| Line 130: | Line 193: | ||
= Open Questions/Issues = | = Open Questions/Issues = | ||
1. How do we verify this statement: "Unlike other sign-in systems, BrowserID does not leak information back to any server (not even to the BrowserID servers) about which sites a user visits." Could use some help from Dev and PM on testable use cases. | |||
2. How does this site/technology fit in? Or is it not applicable to our weekly testing? | |||
http://people.mozilla.com/~faaborg/files/projects/firefoxAccount/index.html | |||