SoC:RoamingSupport: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
No edit summary
 
(29 intermediate revisions by 3 users not shown)
Line 7: Line 7:


=== The Sync Service ===
=== The Sync Service ===
This service will be the control center for sync updates and notifications. Classes who are interested in syncing would implement a observer interface (ex. <code>|nsIObserver|</code>). An interface would register itself based on the type of data that it uses. The Sync Service would handle all of the transactions between the observed data objects and the delta update messages.
A new service will be introduced into the mailnews component, called <code>nsIMsgRoamingService</code>. This service will delegate notifications between the data classes and roaming update messages. There are two paths of data through the roaming service, as illustrated below:


This service would also handle the dispatching of update messages. Ideally, another class would handle the gory details for message creation and maintenance of the delta messages and the full sync message.
<div style="text-align: center">[[Image:Flow_dataObjects.png]]</div>
=== Delta Update Messages ===
Note: A "data class" is a interface that controls anything that we want synced, like an <code>|nsIMsgFolder|</code>.
The idea of storage on either the IMAP or POP3 level involves the use of a complete sync message, and corresponding delta messages for each update since the last complete sync message.


We would like to store these messages somewhere other than the INBOX, and would like to avoid the end-user discovering these messages as much as possible. To hide these update messages in the IMAP implementation, we would probably take advantage of an un-subscribed folder. For the POP3 version, we would set some sinks in the POP3 protocol implementation to process our update message, and leave them out of the users inbox.
The roaming service uses the <code>nsIObserverService</code> to handle passing updates to the data classes. The designated topics for the roaming service can be found in <code>nsMsgRoamingConstants.h</code>.  


To prevent any confusion, the subject of each delta message would be clearly labeled.  
The roaming service also maintains the list of update messages that are stored on either the IMAP folder or the POP3 inbox. A typical set of update messages consists of one full update message, followed by a preset number of delta update messages. This setup is illustrated below:


To prevent potential malicious emails from doing bad things during the update, we plan to sign our messages using a PIN implementation. This setup will be similar to the Google Sync feature.
<div style="text-align: center">[[Image:update_messages.png]]</div>
 
=== Update Messages ===
For containing and controlling the data between installations, update messages are used to package and transport the sync data. All sync information is packed into a file. There are two types of update messages:
 
* Full Update Message - Contains everything that the user has selected to sync
* Delta Update Message - Contains only the changes since the last full/delta update message.
 
Some of the sync data can be transferred as the full data file, like the address book's abook.mab file. However, certain data types like user preferences can not be sync'd this way, and contain an update log listed in the file. To help with unpacking all of the data, a TOC file is created and included with each update message. The format of the TOC file looks like this:
 
  NumItems: 3
  Item0: mbook.ab
  Item1: temp.txt
  Item2: temp2.txt


=== Delivery Options ===
=== Delivery Options ===
Line 28: Line 40:


The full sync message would contain a renamed zip archive (.zim or whatever) to help preserve with storage space. The message would also be marked not to delete on the server side.
The full sync message would contain a renamed zip archive (.zim or whatever) to help preserve with storage space. The message would also be marked not to delete on the server side.
=== Syncing Sessions ===
To help preserve the integrity of the sync, a "sync-log" has been proposed to keep track of all of the changes that have been applied (and probably when they were applied).
Since shutting down and not syncing would probably suck for the average user, I have planned to add a pref that allows forced synchronization at shutdown. Either the user can build and save our build send the message depending on the pref.
For IMAP-synched sessions, a timer is probably not needed, as changes can be applied as new delta/update messages are posted. The same philosophy can be applied to the POP3 setting, applying sync changes based on the biff-update interval.
== TODO List ==
To help keep things centralized, I'm keeping track of things I need to work on here. Any suggestions made by others will be listed here as well.
* Shutdown &amp; Refresh AB Datasource
* Mark Update Messages as Read (IMAP not working right?)
* Fix Purge Update Messages
* Initial/Temporary UI integration
* Sign Update Messages [file follow-up bug]
* Include information to setup a roaming account from a full update message.
* POP3 mailboxes that have a limited download size will fully download the entire update message
** [http://mxr.mozilla.org/mailnews/source/mailnews/base/util/nsMsgIncomingServer.cpp#1737 Use an internal filter].
** [http://mxr.mozilla.org/mailnews/source/mailnews/base/util/nsMsgIncomingServer.cpp#1629 Or read the filter from a file].


== Service Synchronization Notes ==
== Service Synchronization Notes ==
Line 35: Line 67:
** Saved Searches
** Saved Searches
** Newsrc files
** Newsrc files
** Rss subscriptions
** Tag definitions
** Views
** etc.. (add more)
** etc.. (add more)
* When to sync?
* When to sync?
Line 40: Line 75:
** shutdown
** shutdown


== Things to keep in mind: ==
* Make the feature extensible
** A simple interface to inherit?
* Need a way to know that an object has changed
* A way to stream the object to a file, or a file pointer
** for newsrc files, filters, (anything that uses a file)
* Things that aren't stored as a file:
** Account Information
** About everything in account.js
*** Sync tag definitions
** An interface that parses and applies these changes:


----


== Item Synchronization Notes and Ideas ==
== Item Synchronization Notes and Ideas ==

Latest revision as of 00:44, 28 August 2007

The Proposal

My initial idea is to create a new interface that registers itself as a service with XPCOM. This service will allow objects to register (and unregister if needed) themselves for synchronization push and pull operations. This interface will also provide at least a high-level path for wrapping up the data to sync in a sudo-email message or IMAP implementation.

The push operations would provide the object a new set of data (in a to-be-determined data structure) and leave the implementation details up to the object.

The pull operation would request data (again in a to-be-determined structure) from the object into the synchronization service.

The Sync Service

A new service will be introduced into the mailnews component, called nsIMsgRoamingService. This service will delegate notifications between the data classes and roaming update messages. There are two paths of data through the roaming service, as illustrated below:

Flow dataObjects.png

Note: A "data class" is a interface that controls anything that we want synced, like an |nsIMsgFolder|.

The roaming service uses the nsIObserverService to handle passing updates to the data classes. The designated topics for the roaming service can be found in nsMsgRoamingConstants.h.

The roaming service also maintains the list of update messages that are stored on either the IMAP folder or the POP3 inbox. A typical set of update messages consists of one full update message, followed by a preset number of delta update messages. This setup is illustrated below:

Update messages.png

Update Messages

For containing and controlling the data between installations, update messages are used to package and transport the sync data. All sync information is packed into a file. There are two types of update messages:

  • Full Update Message - Contains everything that the user has selected to sync
  • Delta Update Message - Contains only the changes since the last full/delta update message.

Some of the sync data can be transferred as the full data file, like the address book's abook.mab file. However, certain data types like user preferences can not be sync'd this way, and contain an update log listed in the file. To help with unpacking all of the data, a TOC file is created and included with each update message. The format of the TOC file looks like this:

 NumItems: 3
 Item0: mbook.ab
 Item1: temp.txt
 Item2: temp2.txt

Delivery Options

The delivery mechanism should anonymous from the details of the actual storage of the message, wether it be IMAP or POP3. The sync service should be able to pass down something like a file pointer to a temporary sync file. The advantage to storing the updates in an individual file is to keep away from the actual "live" data files, and perhaps create the ability to format the update file in a format of our choosing.

At this stage, we have not determined if the storage options in the delta messages, here are the options I have come up with so far:

  • Store all updates for current iteration in one file
  • Store each update in an individual delta message
  • Store each individual update as individual files in one delta iteration.

The full sync message would contain a renamed zip archive (.zim or whatever) to help preserve with storage space. The message would also be marked not to delete on the server side.

Syncing Sessions

To help preserve the integrity of the sync, a "sync-log" has been proposed to keep track of all of the changes that have been applied (and probably when they were applied).

Since shutting down and not syncing would probably suck for the average user, I have planned to add a pref that allows forced synchronization at shutdown. Either the user can build and save our build send the message depending on the pref.

For IMAP-synched sessions, a timer is probably not needed, as changes can be applied as new delta/update messages are posted. The same philosophy can be applied to the POP3 setting, applying sync changes based on the biff-update interval.

TODO List

To help keep things centralized, I'm keeping track of things I need to work on here. Any suggestions made by others will be listed here as well.

  • Shutdown & Refresh AB Datasource
  • Mark Update Messages as Read (IMAP not working right?)
  • Fix Purge Update Messages
  • Initial/Temporary UI integration
  • Sign Update Messages [file follow-up bug]
  • Include information to setup a roaming account from a full update message.
  • POP3 mailboxes that have a limited download size will fully download the entire update message

Service Synchronization Notes

  • Items to Sync:
    • Address Book
    • Mail Filters
    • Saved Searches
    • Newsrc files
    • Rss subscriptions
    • Tag definitions
    • Views
    • etc.. (add more)
  • When to sync?
    • on a set interval (pref?)
    • shutdown


Item Synchronization Notes and Ideas

Below contains notes for potential hooks for notifying an interface (service possibly) that an object has changed and needs to be synced. These locations might also be ideal for points of entry where the notifying interface could inform the object of changes from an external source.

Address Book

  • Some object that extends from |nsIAbListener|
    • |nsAbDirectoryDataSource|
    • |nsAbView|

Message View Synchronization Ideas

|nsMsgMailViewList| seems like the best place from just poking around. I could just set some sinks in that class for when methods like |AddMailView()| and |RemoveMailView()| are called.

Mail Filters

Saved Searches

News RC

RSS Feeds

Saved Preferences