QA/BrowserID/TestPlan

Overview

Add the BrowserID QA test overview here...

Strategy

Add the BrowserID QA test strategy here...

Scope of Testing

Add the BrowserID QA scope of testing here...

• Client-side: what will and will not be tested
• Server-side: what will and will not be tested

General Test Information

Links and Documentation

• Identity main site: https://wiki.mozilla.org/Identity
  • Production server: https://browserid.org
  • BrowserID Introduction and Demo: https://browserid.org/users
  • BrowserID How-To for developers: https://browserid.org/developers.html

• Services QA Sites:
  • Mozilla QA main site: https://wiki.mozilla.org/QA
  • Mozilla QA main site: http://quality.mozilla.org/
  • Browser Technologies QA: https://wiki.mozilla.org/QA/Browser_Technologies
  • Services QA: https://wiki.mozilla.org/QA/Browser_Technologies/Services

Weekly Test Schedules

• Thursdays: deployment to Production, Beta (QA), and Dev
• Thursdays/Fridays: open testing and experimentation by Dev, QA, and community
• Following week:
  • Monday - Wednesday: QA testing and sign off of current deployment

Weekly Meetings

Weekly meeting notes are kept on the following EtherPad site:

• http://etherpad.mozilla.com:9000/identity-weekly-meeting

Posting Feedback

• Email List: https://lists.mozilla.org/listinfo/dev-identity
• Blog: http://identity.mozilla.com/
• Google Group: http://groups.google.com/group/mozilla.dev.identity/topics
• or https://groups.google.com/forum/?hl=en#!forum/mozilla.dev.identity
• IRC: #identity

Bugs and Open Issues

Ops Bugs

• Found in Bugzilla
  • Classification: Other
  • Product: mozilla.org
  • Component: Server Operations: Labs

BrowserID Client/Server Bugs/Issues

• Add/update issues: https://github.com/mozilla/browserid/issues

Client/Server Test Environment

Beta test environment

• TBD

Prod test environment

• Server-Side (PIA/IP)
  • Acts as the Implementation Provider (IP) until browsers have native support.
  • https://browserid.org
• Server Access (Localhost examples):
  • https://browserid.org/verify: http://127.0.0.1:10000
  • http://rp.eyedee.me: http://127.0.0.1:10001
  • https://browserid.org: http://127.0.0.1:10002
  • Are there any others?
• Client-Side (RP)
  • Acts as the Relying Party (RP) until there are more public clients to test.
  • www.myfavoritebeer.org

Supported OS and Browsers

Operating Systems

• PC: Win 7, WinXP, Vista (in priority order)
• Mac 10.6, 10.7, 10.5
• Linux: Fedora 14/15, EL5/6, Ubuntu

Browsers

• Minimal support would have to include a native browser plus Firefox
  • Windows: IE 8, FF 4/5, Google Chrome latest
  • Mac: Safari latest, FF 4/5, Google Chrome latest
  • Linux: FF 3/4/5, Google Chrome latest

Major Test Areas

Code Verification

Verify non-QA-testable fixes/code changes by direct inspection (as required)

• Server Source: https://github.com/mozilla/browserid
• Client Source: https://github.com/lloyd/myfavoritebeer.org/

Installation/Deployment Verification

QA needs access to Beta test environment in order to verify the deployment to server side (PIA/IP) and client side (RP). It would be helpful for QA to verify the weekly deployment to the server, along with any client-side changes (JavaScript library, JavaScript API). For early trains, this should be covered by information given in the BrowserID deployment ticket (Bugzilla). This would amount to a delta from the previous train that QA can record as part of their testing.

For subsequent trains, the Beta test environment or a QA environment (deployed with the same build) could be used to verify server deployments and client-side changes.

Sanity/Acceptance

Manual and automated testing on client-side to pass a minimal level of acceptance without which, QA testing of BrowserID can not proceed:

Basic Functional

Manual and automated testing on the client and the server to verify basic functionality of BrowserID:

UI

Manual and automated testing on the client and the server to cover all aspects of the current UI:

Bug Verification

Manual testing of bugs/issues resolved for this weekly cycle of testing. Test cases generated during this testing can be moved to an automation tool for bug regression (see below).

Bug Regression

Manual and automated testing of Verified/Closed bugs and issues from previous cycles of testing. Ideally, this would be in the form of a quick automated test/script to check key fixes/changes/updates. QA to work with Dev and PM to come up with a list of bugs/issues candidates to from the test cases for manual and automated testing.

OS/Browser integration

Manual testing/verification of the required OS and browser configurations.

Areas Not Covered

Currently, QA has not automation configured for the weekly BrowserID testing. TBD: Client-side automation TBD: Server-side automation, primarily to cover the API

Setting Up Test Emails

Email-side (PIA)

• This is the Primary Identity Authority (PIA) for all the transactions.
• Examples for testing: Mozilla, Yahoo, Google, ISPs
• Also available for testing are sites like Mailinator

Using Internal Email Accounts

• If secure enough to do so: <mozilla name>+tag@mozilla.com
• Otherwise: Create emails at www.mailinator.com
• Do we have other tested (re: reliable) methods for creating test/dummy email accounts?