Thunderbird:Autoconfiguration
Goal
Objective: Make my non-tech friends use Thunderbird, by making it dead-simple to set up.
Most people are using webmail these days, mainly because it's so easy. You only need to know the URL (usually linked from the provider's homepage) and email address and password, and there's your inbox already. ISPs, although all of them provide POP/IMAP, are leading users to webmail because of that ease of use (and free mail providers do so for the advertizing revenue).
Goal: Setting up Thunderbird should be as easy as download/install and entering real name, email address and password. The Account Setup Wizard consists of only one screen.
Proposal
In the Account Setup wizard, if the "Email account" radio button is selected (which is the default), 3 text fields are visible and enabled: real name, email address and password.
Email address is properly syntax-checked, and the existance of the domain in DNS is checked.
The domain if the email address is used to determine the configuration (POP/IMAP and SMTP server names, SSL yes/no, authentication methods etc.), via several mechanisms:
- The legacy rdf files in <installdir>/isp/, like we have for Google Mail right now. (This may be dropped in favor of the next steps.)
- Try to contact a mail configuration server of the provider
- E.g. define an DNS TXT record or similar on domain example.net (for my.account@example.net) which contains an URL like https://mailconfig.example.net/mozilla.xml .
- That file contains the mail configuration, essentially the same as in RDF files, just the format in normal XML and a bit cleaned up. The email address (before @ or with domain) that the user can be used as placeholder in the config file, so the file is the same for all users (i.e. static).
- If the email provider does not provide the configuration, we try to find it at a Mozilla server
- E.g. https://autoconfig.mozillamessaging.com/example.net/config.xml
- This service will have the configuration for all the major ISPs and email providers, so there's a 90+% hit rate.
- It will not work for company email addresses.
- If a provider disagrees with a setting there, it can override the configuration by simply providing the config server in step 2.
- Other ways, esp. for company email accounts? Avoid heuristics and trail&error!
- If all fails, we ask the user to enter the configuration, using the existing wizard.
If we could find a configuration, we're done already, with that one screen.
The password dialog does not need to come up either, as the password has already been entered by the user and the wizard filled it in using the password manager.
We should also provide an option to ignore the autoconfig and go into manual config, e.g. using an extra radio button "Email account, manual configuration" (this works exactly like the current "Email account" option) in addition to the new "Email account" option which does the automatic config.
UI
+--------------------------------------------------+ | *Account Setup* | |~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| | (o) E-Mail account | | | | Full name: [ ] | | E-Mail address: [ ] | | Password: [ ] | | | | [ ] Manual configuration | | | | ( ) RSS News Feeds and Blogs | | ( ) Newsgroup account | | ( ) Calendar | | | | [ Cancel ] [ Next/Done ] | +--------------------------------------------------+
(unchecked Manual config means auto config; network call status at bottom, left of Cancel button) or:
+--------------------------------------------------+
| *Account Setup* |
|~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
| (o) E-Mail account |
| |
| Full name: [ ] |
| E-Mail address: [ ] |
| Password: [ ] |
| |
| (o) Auto-detect: Status here(*) |
| ( ) Manual configuration |
| |
| ( ) RSS News Feeds and Blogs |
| ( ) Newsgroup account |
| ( ) Calendar |
| |
| [ Cancel ] [ Next/Done ] |
+--------------------------------------------------+
(*) Auto-detect status text:
1. Fill-in the fields above
2. Searching...
3a. Completed (found configuration)
3b. Failed (use manual configuration)
If the user finished (onblur) the email address text field (e.g. while he enters the password), the wizard makes network requests to try to find the configuration. The network activity is indicated by a spinner next to the "Searching..." text.
If we found a configuration, the spinner is removed and the "Next" button turns into "Done". In that case, there is no second page, not even the summary screen. The user ends up directly in his inbox, which was the goal.
If the user checks the [x] manual configuration option, the "Done" button turns into "Next" again. The server names etc. from the autoconfig file are still used in this case, but only to pre-fill the UI fields. That way, an advanced user can see, check and alter the values, but still takes advantage of autoconfig when possible. Open question: What about settings that are in the config file, but not exposed in the wizard? Used or not? May be important, but user can't check them before use.
Config file format
It should be XML, with a clearly defined format, to be stable and usable by other mail clients, too.
The current RDF format seems good in structure, but needs to be cleaned up to remove the RDF bits, Mozilla specifics etc.. It would be processed as normal XML, e.g. using E4X in Thunderbird.
<?xml version="1.0"?>
<clientConfig
xmlns="http://mailconfig.mozillamessaging.com/fileformat/namespace/2008">
<emailProvider id="googlemail.com">
<domain>gmail.com</domain>
<domain>googlemail.com</domain>
<displayName>Google Mail</displayName>
<displayShortName>GMail</displayShortName>
<incomingServer type="pop3">
<hostname>pop.googlemail.com</hostname>
<port>995</port>
<socketType>TLS</socketType>
<!-- unexcrypted, SSL, TLS
"if available" options considered harmful -->
<username>%EMAILFIRSTPART%</username>
<authentication>DIGEST-MD5</authentication>
<!-- anonymous,
plain,
login,
CRAM-MD5,
DIGEST-MD5,
KerberosV4,
GSSAPI (Kerberos v5),
-->
<displayName>Google Mail</displayName> <!-- needed? -->
<pop3>
<leaveMessagesOnServer>true</leaveMessagesOnServer>
<deleteMailLeftOnServer>false</deleteMailLeftOnServer>
<authentication>plain</authentication> <!-- todo list -->
</pop3>
<!-- remove the following and leave to client/user? -->
<loginAtStartUp>true</loginAtStartUp>
<downloadDuringNewMailCheck>true</downloadDuringNewMailCheck>
<rememberPassword>true</rememberPassword>
</incomingServer>
<outgoingServer type="smtp">
<hostname>smtp.googlemail.com</hostname>
<port>587</port>
<socketType>TLS</socketType> <!-- see above -->
<username>%EMAILFIRSTPART%</username> <!-- if smtp-auth -->
<authentication>smtp-auth</authentication>
<!-- none (server grants access based on IP address),
smtp-auth (RFC 2554, 4954),
smtp-after-pop (authenticate to incoming mail server first
before contacting the smtp server)-->
<displayName>Google Mail</displayName> <!-- needed? -->
<addThisServer>true</addThisServer>
<useGlobalPreferredServer>true</useGlobalPreferredServer>
</outgoingServer>
<identity>
<!-- needed? -->
<!-- We don't want Verizon setting "Organization: Verizon"
for its customers -->
</identity>
<!-- Will not be used with this proposal: -->
<sampleEmail>example@googlemail.com</sampleEmail>
<sampleUserName>example</sampleUserName>
<usernameDescription>Google Mail Username</usernameDescription>
<emailProvider>
<clientConfigUpdate url="https://www.googlemail.com/config/mozilla.xml" />
</clientConfig>
Placeholders:
- %EMAILADDRESS% (full email address of the user, usually entered by the user)
- %EMAILFIRSTPART% (email address, part before @)
- %EMAILDOMAIN% (email address, part after @)
- %REALNAME% (needed?)
TODO:
- IMAP
- All settings and enum values
Security
If you are OK with POP3 and plain authentication over the Internet, the following is of no interest to you. :-)
ISP config lookup
To avoid password stealing (DNS attacks are easy), we should verify the authenticity of the configuration file, that it's from the company which owns the email domain. This could be done by downloading the config file via https, but that's only useful when we actually check the domain name (subject) in the certificate against the email address domain that the user entered. That would be as secure as HTTPS on the web, but runs in a problem for email providers with vanity domains like coolpeople.com, they won't buy a cert for each domain.
Mozilla config lookup
If the configuration files on the Mozilla service can be contributed in a wiki-like way by anonymous people, the password theifs could just submit a config file for a big ISP and wait for the passwords to come in. The server should check that the domain of the IMAP/SMTP servers matches the domain of the email address that the config applies to, e.g. config file for aol.com must have something.aol.com as IMAP/SMTP server. That will work for many, but not for those which have several domains goign to the same server (e.g. gmail.com = googlemail.com), so probably there either need to be some automated tests (e.g. checking that both domains are served by the same DNS server and return the same MX entries) or failing that a trusted human moderator.
DNS based discovery of configuration/setup files for mail clients
Requirements Notation
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119].
Overview
Configuration files for all mail servers would be fetched according to the following standard, providing a unified approach and UI, based on existing discovery channels and transport layers.
Definitions
The location of the configuration file shall be discovered by the following method:
1.) The MX record(s) of the user supplied email address is fetched from the DNS zone. In case no MX record exists, the A record of the base domain name is used [RFC 2821].
2.) A TXT record with the content "mailconf=https://" is fetched from the DNS zone based on the highest priority MX record found. If no matching TXT record is found the next lower MX record shall be fetched. At last the A record of the domain is checked for a TXT record.
3.) A sanity check shall be performed (correctness of the URL) and in case of failure proceed to the next possible TXT record as shown above. Only SSL secured locations are possible, meaning the protocol prefix https:// must be present.
4.) When no valid TXT record is found, automatic setup is not possible and aborted.
5.) The first valid TXT record shall be used in order to fetch the configuration file. In case the SSL secured server can not be chained to a trusted root, has a domain name mismatch or other failures, connection shall be aborted. TB may use the next valid TXT record if possible and repeat this step.
6.) TB shall include the originally requested email value as referrer information in its GET headers . The value of the referrer header shall have the format "mailto:user@domain.com". This allows serving of dynamically generated configuration files.
7.) The configuration file is downloaded and checked for sanity and parsed. At this stage the user is present with a UI dialog, stating the location of the configuration file (The actual URL from the which the config was fetched) and present other useful information.
8.) The user may make selections such as different choices (like POP3 versus IMAP or IMAP versus IMAPs etc.). Alternatively TB may use by default best and most secure protocols.
9.) The user confirms his preferences (if any) and confirms the setup of the account.
10.) TB tries to use the new settings and prompt for passwords etc. as usual.
Advantages
1.) Unified standard for all ISPs.
2.) Unified UI implementation possible.
3.) Uses existing infrastructures, protocols and transport layers.
4.) Reasonable secure.
5.) Highly scalable for public Internet and Intranet alike.
6.) Decentralized and autonomous.
Examples
DNS record example:
mta1 IN A 192.192.202.201
IN MX 10 mta1.domain.com.
IN TXT "mailconf=https://someurl/somepath/somefile"
mta2 IN A 192.192.202.202
IN MX 20 mta2.domain.com.
IN TXT "mailconf=https://someurl/somepath/somefile"
Disclaimer
This standard shall be licensed according to Mozilla compatible licenses and is copyright (c) 2008 of Eddy Nigg. Not sure if this is needed, but prefer to be on the save side for now, just in case other vendors want to adopt it. Please advice on this.
Related Information
The newsgroup discussion is at news://news.mozilla.org:119/mozilla.dev.apps.thunderbird (Subject: Proposal: Auto-configuration, Date: 2008-03-04)
There's related information on "Thunderbird ISP hooks" at http://developer.mozilla.org/en/docs/Thunderbird_ISP_hooks.