QA/BrowserID/TestPlan: Difference between revisions

← Older edit

QA/BrowserID/TestPlan (view source)

Revision as of 01:29, 3 July 2013

2,397 bytes added , 3 July 2013

→‎Sanity/Acceptance/Smoke

Edwong

Confirmed users

964

edits

@@ Line 41: / Line 41: @@
 ** https://docs.google.com/spreadsheet/ccc?key=0ArxosXOjCBacdHNrWFdBQmE3RU0xRFFwMUcxQWdRcXc&hl=en_US
-* BrowserID Test Cases: Set 1
+* Test Cases in Moztrap:
-** https://docs.google.com/document/d/1fmT-Egj4-pJeQBKJiHw_eIIOkR-TmW7SE1yeXgbqF3U/edit?pli=1
+** https://moztrap.allizom.org/manage/cases/?filter-productversion=2
+** and here: https://moztrap.mozilla.org/manage/cases/?filter-suite=82
 === Weekly Test Schedules ===
@@ Line 53: / Line 54: @@
 === Weekly Meetings ===
-* BrowserID Team Meeting: Mondays, 9:30am PST/PDT in 2-CYA
+* BrowserID Team Meeting: Mondays, 10:30am PST/PDT in Vidyo Rm: Identity
 ** Wiki: https://wiki.mozilla.org/Identity/WeeklyMeeting
-* BrowserID Bug Triage Meeting: Thursdays, 9:30am PST/PDT in 2-JK
+* BrowserID Bug Triage Meeting: Thursdays, 9:30am PST/PDT in Vidyo Rm: Identity
 ** https://github.com/mozilla/browserid/issues
@@ Line 70: / Line 71: @@
 ** benadida = Ben Adida developer
 ** thunder = Dan Mills PM
-** petef = Pete Fritchman OPs
+** gene = Gene Wood OPs
 ** jbonacci = James Bonacci QA
 ** jrgm = John Morrison QA
@@ Line 225: / Line 226: @@
 ** https://wiki.mozilla.org/QA/BrowserID/BrowserID_Basic_Install#Running_Unit_Tests
 * Instructions are good for any installation...
+=== Automation Testing ===
+* There are two main ways to run automation testing:
+** Directly through the Selenium/Jenkins/Sauce Labs automation environment
+** Locally using a stand-alone Selenium server and the built-in automation tests
+* Selenium/Jenkins/Sauce Labs automation - see the following links
+** https://wiki.mozilla.org/QA/BrowserID/BrowserID_Basic_Install#Running_the_Automation_Tests
+** https://wiki.mozilla.org/Identity/QA#BrowserID_Automation
+*** Selenium BID dashboard: BrowserID dashboard: http://qa-selenium.mv.mozilla.com:8080/
+*** Browserid-tests repo: https://github.com/mozilla/browserid/tree/dev/automation-tests
+*** Dave Hunt's BIDPOM (BrowserID Page-Object Model) repo: https://github.com/davehunt/bidpom
+* Automation on the localhost - see the following links:
+** https://wiki.mozilla.org/QA/BrowserID/BrowserID_Basic_Install#Running_the_Automation_Tests
 === Sanity/Acceptance/Smoke ===
@@ Line 231: / Line 247: @@
 * See the Test Cases (listed in the "Links and Documentation" section above) for more information.
-* Server-side (https://diresworb.org/)
+* Server-side (https://login.anosrep.org/) [https://moztrap.mozilla.org/manage/cases/?filter-productversion=2&filter-suite=82 MozTrap] [https://github.com/mozilla/browserid/blob/dev/automation-tests/persona_server/tests/test_manage_account.py AUTO]
-** Sign Up with a new account and one email
+** [https://moztrap.mozilla.org/manage/case/989/ Sign Up with a new account and one email]
-*** Verify correct email verification sent to correct account (email provider)
+** [https://moztrap.mozilla.org/manage/case/990/ Sign In - Sign Out with a current account of one email]
-** Sign In with a current account of one email
+** [https://moztrap.mozilla.org/manage/case/992/ Change the password on a current account with one email]
-** Sign Out from a current account of one email
+** [https://moztrap.mozilla.org/manage/case/993/ Delete/Remove a current account with one email]
-** Change the password on a current account with one email
-*** Verify correct email verification sent to correct account (email provider)
-** Delete/Remove a current account with one email
 * Client-side (http://myfavoritebeer.org or http://123done.org using one desktop OS with one browser)
@@ Line 252: / Line 265: @@
 ** Delete/Remove the account from the server-side
-* Client-side (http://myfavoritebeer.org or http://123done.org using Android with stock browser)
+* Client-side (http://beta.myfavoritebeer.org or http://beta.123done.org using Android with stock browser)
 ** Sign In using one RP/client to create a new account with one email
 *** Verify correct email verification sent to correct account (email provider)
@@ Line 274: / Line 287: @@
 * Site Access - Stage
 ** Verify use of HSTS (for https:// only sites)
-** Verify that the Stage server site can be accessed through secured HTTPS: https://www.diresworb.org/
+** Verify that the Stage server site can be accessed through secured HTTPS: https://login.anosrep.org/
-** Verify that the Stage server site can not be accessed using only HTTP, but instead gets redirected to the HTTPS site: http://www.diresworb.org/
+** Verify that the Stage server site can not be accessed using only HTTP, but instead gets redirected to the HTTPS site: http://login.anosrep.org/
-** Verify that the Stage RP/client site can be accessed through standard HTTP: http://beta.myfavoritebeer.org/ or http://123done.org
+** Verify that the Stage RP/client site can be accessed through standard HTTP: http://beta.myfavoritebeer.org/ or http://beta.123done.org
-** Verify that the Stage RP/client site gets a certificate warning/error or is blocked when accessed through secured HTTPS: https://beta.myfavoritebeer.org/, http://123done.org
+** Verify that the Stage RP/client site gets a certificate warning/error or is blocked when accessed through secured HTTPS: https://beta.myfavoritebeer.org/, http://beta.123done.org
 ** Verify similar for the following RPs/clients:
 *** carrera.databits.net:9999/
@@ Line 283: / Line 296: @@
 * Accounts and Email Verification
 ** Creating an account from the server site using Sign Up
-** Creating an account inline (at first use of an RP/client Sign In)
+** Creating an account inline (at first use of an RP/client Sign In) [https://github.com/mozilla/BrowserID-Tests/blob/master/123done/tests/test_new_user.py AUTO]
-** Email notifications for new accounts: verification email through email provider with proper email account listed, live verification link, etc.
+** Email notifications for new accounts: verification email through email provider with proper email account listed, live verification link, etc. [https://github.com/mozilla/BrowserID-Tests/blob/master/123done/tests/test_new_user.py AUTO]
 *** Test with emails/accounts on various, popular email servers/services
 *** See more details in the next bulleted list
@@ Line 292: / Line 305: @@
 * Email Notifications for server or RP/Client
 ** On the same OS
-*** Email notification and verification using one browser
+*** Email notification and verification using one browser [https://github.com/mozilla/BrowserID-Tests/blob/master/123done/tests/test_new_user.py AUTO]
 *** Email notification on one browser, verification on an another
 ** Across multiple OS
@@ Line 313: / Line 326: @@
 * Accounts and Emails
-** Adding additional emails to an account
+** Adding additional emails to an account [https://github.com/mozilla/BrowserID-Tests/blob/master/123done/tests/test_add_another_email.py AUTO]
 ** Attempt to add an email (that may or may not be yours) from another account
 ** Deleting one or more emails (without actually deleting the account)
@@ Line 689: / Line 702: @@
 * Server/service scenarios - failover/recover, temp outage, traffic flow, etc.
 ** Zeus/load balancer issues
-** webhead for browserid and verifier
+** webhead for browserid, verifier, router, and static
 ** swebhead for dbwriter
 ** keysigner for keysigner
 ** mysql/db
-=== OPs Load Testing ===
+=== Load Testing ===
-* Load Testing in Stage is handled by OPs using BrowserID code provided by Dev.
+* Load Testing in Stage is handled by QA using BrowserID code provided by Dev.
-* QA should monitor all load tests run by OPs
+* QA should monitor all load tests
 * QA should also schedule specific load tests to cover various requirements for server-side testing.
 ** Work with OPs to queue up specific load tests to check Server stability, various metrics
-* Monitor load tests with OPs dashboards
+* Monitor load tests with OPs dashboards - e.g., http://pencil.scl2.stage.svc.mozilla.com/dash/stage/bid_webhead (requires access to the MPT VPN)
 * Analyze text and visual data for functionality and peformance
 * REF:
 ** load_gen: https://github.com/mozilla/browserid/blob/dev/bin/load_gen
+<pre>
+An example command line to run the load test:
+    nohup bin/load_gen -s https://verifier.login.anosrep.org -o -m 250000 -u 1/250
+-s  -- the URL to the browserid server
+-o  -- don't request static resources
+-m  -- maximum active users to simulate (ADU)
+-u  -- use this range of #@loadtest.domain users (these users already exist in the
+       stage database. If running your own instance, do `CREATE_TEST_USERS=1000
+       BROWSERID_FAKE_VERIFICATION=1 NODE_ENV=test_mysql npm start`).
+I typically run the above command piped into this:
+    | /usr/local/bin/tai64n | /usr/local/bin/tai64nlocal \
+      >> /path/to/load_gen.log-`date +%Y%m%d-%H%M` 2>&1 &
+which will timestamp the output and save it in a logfile, etc.
+</pre>
 == QA Sign-Off for Stage ==