View source for QA/Thunderbird3/TestInstructions/Alpha3

Thunderbird 3 Alpha 3 Release Candidate Test Instructions
__TOC__

=Overview=

Release Candidate builds for Alpha, Beta and ... provided by builders must be tested by volunteers and the QA team according to a rigorous process in order to certify that a release candidate build is worthy for public release.

We have just a few days to complete testing, so we need all available hands.  Thank you for your interest and (we hope) your enthusiastic participation. As always, if you need help, it is available on [[QA/Thunderbird3/TestResults/Alpha3#Contact_Info|IRC and in the newsgroups]].

We have several types of tests (most of which utilize litmus), and you can choose which to run based on your available time.  BFT testing and l10n, and Mac platform tests are our greatest need.

* BFT (Basic Functionality Tests) - touches most functional areas. If you do not have enough time to do BFT tests we hope that you will ...
* Smoketest - touches a handful of functional areas (a much reduced BFT)
* Spot check - a visual "look see" 
* l10n (localization) - A set of litmus tests just for localized builds. If you can test a localized version of Thunderbird, we would appreciate you doing so.

If you can't help before the scheduled testing period ends, please stop back and run the tests as soon as you are able.

You are not required to run a full test set to completion. So it's OK if you don't have much time or think you might run out of time. In fact, we'd rather have you do a short amount of good testing than no testing at all. Tests done using litmus get recorded as you go through the list, so stop whenever you need ... and return later when (and if) you are able. Leave the test session open in the browser, or remember which test you did last, then start at the next step after starting up the litmus test process when you return.

=Details=

'''Important'''
* Use '''only the specified builds. Do not use a nightly build, nor a previous alpha or beta.  We are qualifying on the a specific set of builds.
* Please file a bug whenever you encounter a problem.
* If a litmus testcase:
** Is not run (you skipped it) - please do not mark it in any way with a comment, passing or failed.
** Fails - please comment in the test, file a bug, and include URL of testcase in bug
** Is unclear - please comment with steps and wording you think would make it more clear
* [[QA/Thunderbird3/TestResults/Alpha3]] documents the release details. You want to check sections 1.1 to 1.6, '''especially the release notes.''' Contact methods are also listed. 

==Builds==

Builds to be tested are listed in the [[QA/Thunderbird3/TestResults/Alpha3|release details]]. Use a localized version if possible. 

==BFT==

Basic Functionality Tests is organized into subgroups listed below. It is a large list, but to shorten your testing time (running the full set takes 4-6 hours) you can focus by subgroups as follows:

* <font color=red>'''*''' indicates priority items</font> - if you have the resource required please do '''*''' groups before others
* '''All''' indicates items we hope everyone will test
* '''AL''' and '''MZ''' - indicates please run that group if your last name/surname is in the range A-to-L or M-to-Z

If you start running out of time and think you won't be able to return (or have time left after doing the assigned subgroups), you might return to the litmus menu "Select a Testgroup and Subgroup to Test" and pick one or two subgroups that show the least amount of testing coverage.

Getting started with litmus:
# You've already read the [[QA/Thunderbird3/TestResults/Alpha3#Release_Notes|release notes]], of course.
# Login to litmus.mozilla.org (create an account if you don't have one)
# Visit https://litmus.mozilla.org/run_tests.cgi?test_run_id=34
# Complete all information about your setup - paste buildid, set platform, OS, and locale (note: there is only one linux OS selection). NOTE: help|about will not give you buildid. builid must be 10 or 14 digits of the form YYYYMMDDHH or YYYYMMDDHHMMSS. Two ways to get buildid:
#* copy to clipboard using NTT after installing [https://addons.mozilla.org/en-US/thunderbird/addon/6543 Nightly Tester Tools extension]
#* application.ini file in Thunderbird program directory (mine for windows is "20081006095651")
# Click "submit configuration"
# You're off and running the tests! For BFT please do '''*''' items first. Plan 2.5-3.5 hours to do half the list (as assigned)

 '''Subgroup assignments for BFTs'''
 &nbsp;
   All-Install, shutdown, uninstall
    MZ-Migration (from other mail systems)
    MZ-Import (from other mail systems)
    MZ-Window configuration
   All-Toolbars and menus
   All-Account settings & Preferences (Options)
    <font color="red">*  IMAP accounts</font>
    <font color="red">*  POP accounts (exclude Global Inbox)</font>
    <font color="red">*  Global inbox</font>
   All-Mail composition
    MZ-Spell checker
    MZ-RSS account & subscriptions
    MZ-Newsgroups
   All-Navigating and displaying messages
    AL-Downloading and saving
    AL-Image blocking
    AL-Return receipts
    <font color="red">*  Proxies</font>
    MZ-Offline, disk space
   All-Moving, copying, deleting messages
   All-Views and labeling messages
    MZ-Message filters
    MZ-Message search
    AL-Address search
    AL-Saved Search folders
    AL-Message Grouping
   All-Message Aging
   All-Quicksearch
    MZ-Address books
    MZ-Junk mail
    MZ-Extensions
    AL-Theme management
    AL-Help
    AL-Printing
    AL-Master Passwords & password management
    <font color="red">*  Phishing, spoof detection</font>
    <font color="red">*  Secure connections</font>
    <font color="red">*  Digital signing, encrypting messages</font>
    AL-Software Update

==smoketest==
Access litmus using the "Getting started" steps above for BFT, but use the smoketest URL: https://litmus.mozilla.org/run_tests.cgi?test_run_id=33. Should take about 1.5 hours. (Ignore "Subgroup assignments" listed above - it is not relevant)

==Spot check==
Spot check takes little time because is just a casual visual inspection and does not require the use of litmus. Install, poke around, inspect, test and try what you can, then visit [[QA/Thunderbird3/TestResults/Alpha3#Spot_Check|spot check results]] and update the wiki.

==locale==

You can do one of the [[QA/Thunderbird3/TestInstructions/Alpha3#Details|tests described above]] using localized build, then update the wiki at [[Thunderbird3/TestResults/Alpha3/L10N]] to help us track which locales have been tested. 

Also, you can run the [https://litmus.mozilla.org/run_tests.cgi?test_run_id=32 l10n trunk baseline tests] that focus on checking that localized language strings are correct. The results are recorded in litmus.