QA/Execution/Web Testing/Project Checklist

Purpose of this document

This is meant to be a quite-literal checklist for determing "go/no-go" status for your website. Where it doesn't apply, feel free to use it rather as a discussion point, but be sure all points are raised and noted.

Other Wiki Resources

• Master list of other WebDev guides: https://wiki.mozilla.org/Websites/Guides

Browser compatibility/coding standards

• Get the list of supported browsers somewhere in writing (srsly)
• All supported browsers look "pretty good" and are 100% functional
  • If not, exceptions are noted and called out prior to shipping
• Team has gone through https://wiki.mozilla.org/WebDev:FrontendCodeStandards

Video Support

• What are the supported video formats?
  • Appropriate fallback in place and working?
    • No Flash
    • Firefox 3.6+
    • IE 7/8/9
    • Opera (latest)
    • Chrome (latest)
    • Safari (latest)
    • Latest Firefox beta

General

• If this is an outsourced project, a test plan and the test cases that the consultants used to validate their work should be included in the hand-off process.
• In URL paths, / and "" (without the trailing slash) both resolve to the same URI/resource
• HTML validates: http://html5.validator.nu/ or http://validator.w3.org/
  • Exceptions can be made for social-network/sharing sites, that may have malformed tags or markup
• Is the robots.txt file configured correctly to prevent bots from spidering to nonsensical pages? (example)
  

RSS Feeds

• Validate: http://feedvalidator.org/
• Links all work
• l10n: other locales look fine (special chars, Unicode, etc.)
• Correct timestamps

Logged-in vs. logged-out functionality

• Ask/test what the difference between the logged-in vs. logged-out functionality is
  • It's probably quite a bit more than you expected

Social media

• Facebook
  • Private or public? If private: sandboxed, or test accounts?
  • Test with "Secure browsing (https)" (on https://www.facebook.com/editaccount.php?ref=mb&drop) enabled and disabled
• Twitter
  • Enable "HTTPS Only | Always use HTTPS" (https://twitter.com/settings/account)
  • Length of pre-populated tweets in en-US and other locales (l10n)
  • Unescaped vs. escaped characters (string literals)
  • Unicode chars
• ShareThis, etc.
• QR Codes
  • Try different readers (AT&T Scanner, QuickMark, QR Droid, etc.)

Performance/load testing

• Doesn't tank on YSlow! (Strive for an A with ruleset v2)
  • CDN is hooked up, if needed (static images, CSS, JS, videos) -- see "Site Config" section, too, below
• Load-tested, if needed? Make sure to bring this up

Accessibility

• Is the site generally accessible?

JavaScript-disabled

• Does the site support JavaScript disabled? If so, where?
  • What's the user messaging?

Security

• Complete the following (taken from WebAppSec/Secure_Coding_QA_Checklist)
  • Test: Input Validation For User Controlled Data
  • Test: SQL Injection
  • Test: Output Encoding For User Controlled Data
  • Test: CSRF
  • Test: Account Lockout -- INACTIVE
  • Test: X-Frame-Options
• Runs on both HTTP / HTTPS? Mixed-content warnings? Cert set up?
  • Should HTTP requests get automatically redirected to HTTPS, by default?
  • Check to see that third-party assets use relative paths, where possible
    • e.g. Webtrends tags, social media, etc.

Project/release-cycle

1. Has a Webdev bug, filed from https://intranet.mozilla.org/webtools/authenticate/login
2. Has a test plan (which can augment this, but lists specific bugs, milestones, project owners, etc.)
   1. Reviewed by team
3. Has clear bug-filing/tracking, either in a new Bugzilla component, existing (with whiteboard, etc.), or in Basecamp
   1. If in Bugzilla, has clear milestones
4. Listed on https://wiki.mozilla.org/QA/Execution/Web_Testing#Current_Projects
5. Regular status meetings
6. IRC channel?
7. If needed, a tracking bug with dependencies set for other groups' work (IT, Webdev, Marketing, etc.)
8. Make sure to schedule a brief meeting where all parties (Marketing/Webdev/QA) are present and discuss the open issues, and any last-minute (but necessary) changes
9. Ask for PRDs, user-flow diagrams, testing notes
10. Are the third-party developers cc:d on all relevant bugs? If possible (i.e. not "mozilla-confidential" flagged), have them follow the Bugzilla component (and make sure they have accounts).
11. Has a push bug, and a release checklist, even if you don't think they need it (they do)

Optimal Bugzilla Workflow

Each project can be different, but please use this flow if starting a new project:

• A bug is filed in the correct product/component
• Bug is triaged for severity, priority and assigned to a target milestone
• Bug is fixed in work based on next target milestone release
  • Dev puts bug# in git comment
  • Dev puts commit url in bugzilla comment
• Dev marks bug RESOLVED:FIXED
• Tester verifies bug on stage, bug marked RESOLVED:VERIFIED
• Code is deployed to production
• Tester verifies bug
• Tester verifies deployment/push/server-ops bug

Devs may fix bugs before triage, they should update Target Milestone to the next release.

Whiteboard may be used for various reasons:

• [good first bug] - Communicates a contributor opportunity

Site Config

• Make sure you get Django-traceback emails, on both prod and trunk/staging
• Any vanity domains? Registered/set up, and working?
• Has a favicon? That works in IE, too?
• reCaptcha APIs on staging?
• Google Maps API key on staging?
• Metrics code is correct/working for the site (typically Webtrends)
  • Ask Blake Cutler or Laura Forrest (preferably Laura)
• Any rewrite/redirect rules are in place and working?
• Is the site planned to be hooked up to a CDN? Is it set up for testing on staging, and without any cert-signing problems?

Tools

• You're using as many of the tools as appropriate
  • Suggested:
    • PowerFuzzer
    • NetSparker
    • Xenu
    • XSS Me