Thunderbird:Testing

<< Back to Thunderbird Home Page

QA is a big space and there are many ways to help. Whether you have no experience or lots of experience, you can help make Thunderbird better. Your participation is needed and welcomed. Join in, and you too can feel you contributed to the new release of Thunderbird.

You are encouraged to have a look at the items below, but if you want a quick chat about it, please visit IRC #tb-qa. You might also be interested in the ongoing QA Team Interviews.

If you need other types of information about Thunderbird such as development, releases, add-ons, building, plans, etc, please visit Thunderbird wiki and Mozilla Messaging.

How You Can Help in Thunderbird QA

Ways you can help maintain and improve Thunderbird's quality:

• participate in community testing: Bug Days, Test Days, QA New Release Candidates of upcoming versions, i.e. point releases, Thunderbird 3 (3.0, 3.1 aka 3.next, etc), betas, and/or alphas.
• run nightly builds and early releases on a daily basis
  • Nightly testers tend to hang out in Thunderbird Builds forum
  • How to: get a stack off a debug Thunderbird hang
• stay informed of QA activities and announcements
  • Talk to and get help from other testers in IRC channel #tb-qa. IRC Instructions
  • subscribe to thunderbird-testers announcement list
  • mozilla.dev.apps.thunderbird newsgroup, web, mailing list
  • mozilla.dev.quality newsgroup, web, mailing list
  • Planet: subscribe to Mozillamessaging's Planet, moco planet
  • QMO
• watch in real time:
  • bug activity via IRC #tb-bugs (pick a nickname, then click Go)
  • bug activity via bugmail (and help triage them)
  • comm-central patch/fix checkins via RSS
• Participate in Patch Love
• help with Tests, QA support and Documentation
  • develop MozMill Tests
  • improve Litmus Tests
  • improve Documentation and Procedures
  • help integrate Thunderbird QA into the QMO initiative
  • offer to lead or co-lead any of the QA areas, and suggest new QA initiatives
• Report Bugs
• Crashes:
  • If you crash, please fill in "Add a comment" before clicking Crash Reporter's "Restart".
  • Don't like crashes? Help monitor, report, track and recreate/resolve crashes
• contribute suggestions, participate in planning and development by:
  • editing any wiki page that is unclear or that can be improved
  • posting in mozilla.dev.quality newsgroup , web, mailing list
  • Discussion of development efforts in mozilla.dev.apps.thunderbird dev newsgroup

General Information:

• How to ask good questions
• Bug writing guidelines
• Bug triage
• Bug queries
• Getting Involved in Mozilla Messaging
• Thunderbird 3

Triage references:

• Performance Problems, Memory Usage
• Thunderbird:Testing:Shutdown_Hang
• Thunderbird:Debugging_Gloda

Bug Days

• Help triage, improve and reduce the bugs reported to our bugzilla database. Bugday information. Previous / planned QA days.
• Suggest bug groupings to be triaged in Thunderbird:Bugdays_Planning

Test Days

Test new features, specific features, or new releases Testdays information. Previous / planned QA days.

Patch Love

Give love so that patches get checked in, and patch authors stay involved in Thunderbird.

QA Wanted

Some bugs need more work to be usable for our developers. Those bugs are marked with the Keyword qawanted. The idea there is to figure out a way to reproduce, find a proper test case and then add that information to the bug.

QA New Releases

Release candidates for new versions/releases are run through QA (Quality Assurance) testing to ensure we ship high quality code. To properly QA a new release we need many volunteer like you from each OS. To be notified of testing opportunities, please sign up for the testers mailing list and/or you can set up to watch wiki pages at Upcoming / In-progress New Releases | Past releases.

3.0 RC1 builds are available for testing. Please file any new bugs found during the testing period as blocking on bug 524434.

Daily Testing

If you can handle some risk**, a fantastic way to help speed development and improve the Thunderbird's quality is to continuously run an early release or nightly build and submit bugs for problems that you find. And talking to other nightly users is easy.

** Ability to "handle the risk" means you have great backups of your profiles and mail store, and have some tolerance for risk. Dataloss is rare, but is a possibility. Stability is amazingly great, but not guaranteed. And fixes to bugs in these builds don't come overnight. But many people run unsupported builds. Nightlies and some early releases are code named Shredder. Shredder is an unbranded state of Thunderbird code base. Development code eventually ships as a future version, like 3.2.

Some extensions won't work. But many like enigmail and calendar work if you use the right download for trunk or nightly. Many work if you override the version check and carefully test. Determine one at a time which work using Nightly Tester Tools extension. You can also run the current "production" version of Thunderbird at the same time.

See what fixes have checked in at comm-central and watch the changes via RSS. Build forum is also a good source of information.

Gristmill/MozMill Tests

Gristmill/MozMill is a tool to use for UI automation in any mozilla platform developed application. It can provide Thunderbird with the capability to quickly repeat tests in a consistent manner, being another aid to avoid faults creeping into the application with the constant development.

Gristmill is still being developed, however the basic concepts are there and it is possible to run tests manually, see this link for an example within Thunderbird.

The Gristmill scripts are written in Javascript, there is a tutorial available and it should be noted that you do not need an in-depth knowledge of Thunderbird's internals.

Gristmill is still new, and we'd be interested in getting some basic scripts together and setting up some kind of testing architecture. If you're interested, please contact User:Standard8 direct.

We are currently putting together a more detailed, MailNews specific page for Gristmill.

Litmus Tests

Litmus is used to organize and assist human (i.e. non-automated) testing of mozilla software products. We need your help to:

• add new tests - pick from the list of bugs with in-litmus? flag
• fix unvetted v3x tests reported to be "unclear", sorted by summary/title
• run tests (you can run these tests at any time)
  • 1.9 Branch - The baseline test plan for Thunderbird 3.0 builds.
  • 1.8 Branch - The baseline test plan for Thunderbird 2.0 builds.

• • Trunk - The baseline test plan for Thunderbird accessibility and localization L10n.

• • Work in progress tests

If you need more information, please post on IRC or in newsgroup.

Documentation and Procedures

Most of our documentation is on Mozilla Wiki in a few locations. Anyone can edit and improve these pages, you just need an easy to obtain wiki account.

With an account, you can also watch for updates to pages that might interest you. Some wiki pages, like QA days, change for every new date ... so watch is a great notification mechanism.

QMO

QMO, quality.mozilla.org, is a new facility providing integrated QA environment. QMO are preparing to roll out this functionality for Firefox, but for Thunderbird to leverage the QMO infrastructure, we need volunteers to step forward to review, plan and build the Thunderbird specific pieces.

Test Automation

Ideas for Thunderbird's unit test framework can be found here.

Automated testing for Mailnews