QA

From MozillaWiki
Revision as of 17:07, 29 June 2005 by Marcia (talk | contribs)
Jump to navigation Jump to search

Welcome to Mozilla Foundation Quality Assurance (MoFo QA)!

You can contact us by either email or on the #qa channel of irc.


Who We Are

There are thousands of Mozilla contributors who download and test nightly builds of Firefox, Thunderbird, Camino and the Mozilla Suite. The core MoFo QA team consist of:

Asa Dotzler (asa)

QA lead, community lead.

Tracy Walker (tracy)

QA engineer, smoketest guru.

I have been regression testing Mozilla related products since the summer of 2000. Currently, I am on contract with The Mozilla Foundation to perform smoketests on daily builds, run BFT's as needed and help update, upgrade and upkeep our test suites. Most work hours, I can be found on irc.mozilla.org in #qa (tracy).

Jay Patel (jay)

Talkback guru.

Marcia Knous (marcia)

Project Manager and QA contributor. I mostly work on the Mac platform.

Bob Clary (bc)

Test monkey

What We Do

We currently focus most of our efforts on the Firefox and Thunderbird products. We also work with other projects such as Seamonkey to assist their QA and development teams so that we can maximize resources —such as Bugzilla and Testrunner— as well as minimize duplicated efforts.

Smoketests and BFT's

We smoketest the nightly builds of Firefox and Thunderbird (and sometimes Seamonkey), where smoketests consist of the bare-acceptance/sanity tasks of a product. We run basic functional tests (BFT's) at key points during a project cycle, notably before milestone (alpha/beta, final, etc.) releases, which are broader in scope than smoketests. The aim of a BFT is breadth, not depth, of scope, where as many of the features of a given product are touched.

Verifications, ad hoc usage, regressions

The majority of bugs filed result from ad hoc usage. Verifying such bugs is a great means of more deeply exercising the application as well as a useful way to find regressions.

Localization (l10n)

From time to time we have certified localized builds of Firefox and Thunderbird. While this might change (especially with Mister), we have a brief list (for reference) of our Quick QA l10n Checks. However, please refer to the L10N home page for further information.

Test development

Just as developers need to create, modify and maintain code, we in QA need to write, update, revamp and recreate (as needed) test plans and test cases —to ensure that what we use, test and investigate in a given application is correct and current! At present we do most of this manually, but are concurrently investigating automation tools for more repetitive, high-level tasks.

What We Use

We typically use nightly optimized (non-debug) builds for daily usage. However, we also use the release builds (of course!), as well as older builds when trying to narrow down regression windows.

  • For nightly builds, check out any of the mirrors, then drill down to the <product_name>/nightly/ directory. While you can go to <product_name>/latest-* directories, the problem there is that you don't necessarily known when those builds were made. It's best to access the specific build-date directory (e.g., 2005-03-17-08-trunk), to know what you're grabbing.
  • For older builds not listed in the mirror pages, check out the archives at http://archive.mozilla.org/pub/
  • For release builds, simply go to any of the mirrors and drill down to <product_name>/releases/ and select the appropriate directories for version, platform and locale.

Bugzilla

We depend on Bugzilla for filing and tracking bugs and features. We frequently use the query tools, both the "Advanced Search" and "Find a Specific Bug" queries. With the bug count reaching 300,000, there are a couple ways to see what's been frequently reported and duplicated:

Testrunner

We use Testrunner at http://testrunner.mozilla.org for test development and execution of various types of test runs like smoketests and basic functional tests (BFT's). To view the following test plans you need a Testrunner login.

Talkback

When an application crashes, we use Talkback to examine the crash information. A publically available Talkback server can be accessed at http://talkback-public.mozilla.org

There are a number of tools available there:

  • Reports (http://talkback-public.mozilla.org/reports) - Browse topcrash data for all Mozilla products. Topcrash lists, stack traces, user comments and urls are available to help users reproduce crash bugs.
  • FastFind - Lookup individual crash incidents.
  • QuickSearch - Run customized queries to get a better understanding of specific crashes.

How To:

ToDo List:

  • Clean up and update Talkback reporting system and website
  • Create Talkback/release database to track builds
  • Bug fixes and enhancements for Talkback FastFind/QuickSearch tools.

Development tools

We also use several development tools for tracking changes, especially useful for narrowing down regression windows!

QA Resources

Not to be forgotten, both the Bugzilla and main Mozilla websites have lots of QA resources.


Retrieved from "https://wiki.allizom.org/index.php?title=QA&oldid=11349"