Mozilla2:Profile Sharing
Proposal: A Plan for Profile Sharing
Overview
The purpose of this document is to outline a plan for enabling the sharing of files in the user profile by multiple mozilla-based applications running on the same system. Profile sharing has two main goals:
- To enable embedders to write Mozilla-based applications without having to consider profile details. This is something embedders of Trident (MSHTML) take for granted.
- To allow Mozilla's applications to share specific preferences, certificates, and possibly other profile resources. For example, the Forumzilla extension to Thunderbird might benefit from being able to access the cookies stored in a Firefox user's profile.
Long-term, other goals include making it possible to launch two instances of Firefox and have each instance use the same profile. This requires Firefox to share all of its profile contents, which will be costly to implement. It's unclear whether Thunderbird would ever wish to support full profile sharing with other instances of the Thunderbird application.
A separate, long-term goal is to enable profile sharing across hosts. This is a much more complex problem because it involves coordination between different machines. This proposal does not address the problem of sharing profile data across multiple hosts.
Requirements
The major requirements for profile sharing include:
- Enable sharing of profile data between multiple Mozilla-based applications running on the same system.
- Store shared profile data in a directory that is independent of application specific profile data. This directory should only contain files that are shared by applications participating in this profile sharing scheme.
- An application should be able to choose the location of the shared profile directory. This is a requirement in order for profile sharing to be per profile.
- Access to the shared profile directory must be gated exclusively by this profile sharing scheme in order to ensure the integrity of shared profile data.
- An application should be able to choose the profile data it shares. Moreover, in the case of shared preferences, an application should be able to choose the set of preferences it shares with the platform. This level of granularity may be appropriate for other shared profile data as well.
- Profile sharing should not mandate the creation of new file formats for the shared profile data.
- The profile sharing scheme must be frozen to a degree sufficient to ensure compatibility with future applications.
The existing Mozilla-based applications store a number of different files in their profiles. Many of these files are application specific. Some, however, can be generalized to the web-platform, and therefore are likely of interest to other applications built on the Mozilla platform. These are the files that we are most interested in sharing.
Cache
The web cache is by far the largest collection of files stored in the user profile. Because of its large footprint on disk, sharing the disk cache is advantageous since it avoids duplicating a web cache for each Mozilla-based application. NOTE: Thunderbird does not enable the disk cache for mail, but extensions like Forumzilla might benefit from Thunderbird having access to a shared disk cache. Moreover: Seamonkey mail uses the disk cache for HTTP content loaded into the mail frame. This gives Seamonkey mail an advantage over Thunderbird when a user revisits a piece of HTML mail.
Certificates
It is common for corporations to provide certificates to employees for use to sign and seal emails as well as to access internal websites. Therefore there is need for Firefox and Thunderbird to share certificates to avoid having to manage separate certificate databases.
sharing may even be seen as a barrier by some enterprises upgrading to the new apps from Seamonkey or maybe IE+OE (IE and OE access certificates from a registry entry managed by the operating system).
The NSS team is currently working on prototypes of shared databases using SQLite.
Cookies
Cookies are an important part of the web-platform. By supporting shared cookies, we enable other applications to inherit user identity when they access web sites. This could be used by Forumzilla, for example, to make it so that the view of slashdot.org is properly configured per the user's preferences for that site as configured using Firefox.
Preferences
Sharing preferences is perhaps the main use case for profile sharing. The standard example is a user's proxy settings which all Mozilla-based applications need in order to connect to the internet. Allowing users to avoid having to re-enter these settings for each application would be very beneficial. Moreover, when the user needs to make changes to the proxy settings, the user will appreciate the fact that those changes will be reflected to all Mozilla-based applications. An independent options dialog for shared preferences might make sense, but this is probably a detail best left for another proposal ;-)
What About Other Files?
Most of the other files in the profile are application specific. For example, bookmarks, history, address book, etc. are all very specific to individual applications. Some files, like the fastload file and localstore.rdf are also application specific since their contents will depend on the application that generated them.
In the future, it may be desirable to share some of these files with other similar applications or with other instances of the same application. But, again the primary focus of profile sharing is to support the Mozilla toolkit. And, that means starting with sharing the web-platform specific profile data.
Note from bsmedberg:
to reduce the app-author-barrier to shared profiles, I intend to provide a RDF datasource that does sharing. This will probably not be my fancy database-driven datasource, but something that uses a simple N-triples format and keeps all the data in memory.
The IPC interfaces should be frozen for 2.0
darin: Agreed.
A Scheme for Profile Sharing
The IPC Daemon and the Transaction Manager Extension
The IPC daemon was developed for the purpose of being used to enable profile sharing. It serves as a central process that clients can connect to in order to communicate with one another or to communicate with code running in the IPC daemon itself. The IPC daemon is extensible via a simple "plug-in" API
The main benefit of the IPC daemon is that it can be used to synchronize access to shared profile data. It provides basic locking functionality that does not depend on filesystem or operating system locking primitives (that are known to be buggy in some cases). Moreover, via the manager IPC daemon extension, the IPC daemon can be used to actively synchronize in-memory and in-process copies of databases.
Details on the transaction manager design are available here. The transaction manager is a good solution for the problem of sharing preferences and cookies since it minimally perturbs the backends for those modules. The transaction manager API provides synchronized reading and writing as well as notifications about changes to the databases.
A client of the transaction manager simply attaches itself to a named queue, waits for permission to get read access to the corresponding files in the profile, reads those files, and then listens for change notifications on that queue. When the process makes changes to its copy of the database, it posts those changes to the queue so other processes can update their in-memory copies. When a process wishes to write its in-memory copy to disk, it again requests permission from the transaction manage and upon receiving permission it writes the files out to disk. The transaction manager makes sure that a client requesting write access receives all changes to the database before granting it write access. In that way the transaction manager ensures integrity of the serialized databases and protects against data loss. Inotherwords, an application could not inadvertantly stomp over the data set by another application unless it ignored notifications from the transaction manager.
The transaction manager's protocol is very simple and is not at all profile specific. It can even be used to synchronize memory-only databases that are not persisted to the profile.
The disk cache cannot be shared easily or efficiently using the transaction manager. We really have two choices when it comes to sharing the disk cache. We could develop an IPC daemon extension that manages the cache's data structure, with a protocol that clients use to gain read or write access to particular sections of files on disk (recall that some disk cache files are stored in the block files).
Another solution for sharing the disk cache involves utilizing the IPC daemon only to manage reader/writer locks. Then each client process would acquire the appropriate lock(s) and proceed to access the data structures of the cache directly. This probably involves mapping and unmapping the _CACHE_MAP_ and _CACHE_00X_ files frequently. It remains to be seen if this would be prohibitively costly in terms of performance. This solution also locks down more of the cache format since future applications will need to store files in the cache in a format that older applications can still understand.
The IPC daemon module solution doesn't have as many constraints since we can hide many of the details of the cache data structure behind an IPC protocol. This may be enough of a reason to go with this kind of solution.
The NSS .db files can be shared by creating a library with the name
SHLIB_PREFIX"rdb."SHLIB_SUFFIX. NSS will open this library
and look for a symbol named rdbopen. It should have the
following signature:
DB *rdbopen(const char *appName, const char *prefix, const char *type, int flags);
The only description of this API that I could find (outside of the source itself) is here.
The current thought is to use SQLite. This removes the need for any central server, or the use of IPC to talk to it. The NSS team has a prototype.
Next Steps
- Agree on general plan for profile sharing
- Agree on location for shared profile data. Maybe something like this:
~/.mozilla-common/<profile-name>/
~/.firefox/<profile-name>/
~/.thunderbird/<<profile-name>/ - Bring shared preferences and cookies online first (target 1.8 easily).
- Bring shared certificates and cache online next (post 1.8 likely).
- What about ensuring that the cache is stored locally? Need system API help.
Note from bsmedberg:
We need to specify how profile locking and profile sharing interact. In particular, can the profile lock files contain "locator" information to other apps that are already running which need to share profile information?
Example:
- One instance of firefox begins running. There is no profile.lock, so it writes an "IPC locator" into that file: "socket: ./sharing.socket" or something like that.
- A second instance of firefox is started. It sees that profile.lock already exists, so it reads the file, and tries to use the IPC mechanism to talk to "socket: ./sharing.socket".
By adding new "locator" types in the future we could support special situations like a central profile server, or multiple machines sharing a profile through TCP sockets.
darin: This sounds like an excellent idea to me.
What about using DBus at least on *nix platforms for ipc? Note that a win32 port also exists (though not maintained) and the license is similar to X/BSD.