Confirmed users, Bureaucrats and Sysops emeriti
291
edits
User:Gal/SyncDataModel: Difference between revisions
→Bookmarks
| (7 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
== Data model == | == Data model == | ||
=== Assumptions === | |||
We assume that all changes to the local database occur in a mostly-connected state. In other words, people don't change passwords or bookmarks without network connectivity. However, there can be brief network interruptions etc. The service is assumed to be reasonably available (we don't need many 9s, but a few). We use push notifications to shorten the collision window where we care (passwords, bookmarks), and we tolerate collisions where we don't care. | |||
=== Passwords === | === Passwords === | ||
| Line 21: | Line 25: | ||
The entire bookmark tree is represented as one doc. The tree consists of folders ({ description, title, children }), separators ({}), and bookmarks ({ title, uri, description, loadInSidebar, tags, keyword}). We expect dozens of bookmarks, and many hundreds worst-case. | The entire bookmark tree is represented as one doc. The tree consists of folders ({ description, title, children }), separators ({}), and bookmarks ({ title, uri, description, loadInSidebar, tags, keyword}). We expect dozens of bookmarks, and many hundreds worst-case. | ||
'''Key:''' "bookmarks" | '''Key:''' "bookmarks" for desktop bookmark tree, "mobile" for mobile bookmark folder | ||
'''Value:''' JSON of tree as described above. | '''Value:''' JSON of tree as described above. | ||
| Line 51: | Line 55: | ||
== Revision purging == | == Revision purging == | ||
Clients keep the current revision of a doc in shadow couchdb. Since clients don't directly replicate with each other, there is no need for them to keep any more history than that. The server keeps the revision known to the client that is the furthest behind in replication. | Clients keep the current revision of a doc in shadow couchdb. Since clients don't directly replicate with each other, there is no need for them to keep any more history than that. The server keeps the revision known to the client that is the furthest behind in replication. This is needed in case we ever want to do any kind of merging of docs. | ||
== Notifications == | == Notifications == | ||
Clients that have sync enabled should use our notifications protocol to listen to server changes. For bookmark and password changes, we | Clients that have sync enabled should use our push notifications protocol to listen to server changes. For bookmark and password changes, we replicate to the server after a brief delay. For history updates, we group updates with a certain (not necessarily very short) time delay. When the server receives any changes, it will send a push notification to all clients that are currently registered and are known to be out of date. CouchDB maintains sufficient state as part of the replication protocol to find these clients. When clients start, they should also contact the server to receive updates (and register for push). | ||
== Optional CouchDB protocol modifications == | == Optional CouchDB protocol modifications == | ||
=== jsondiff === | |||
CouchDB is not an overly wire-compact protocol. This is particularly painful for bookmarks updates since we repeat a lot of data that is unchanged from the last state. Instead of sending the whole document again, we should instead do a jsondiff and send the diff only. The server and the client both have the last _rev and can reconstruct the full version. Once the initial version of this is operational, we should consider a fairly simple addition to the CouchDB protocol to make it more compact on the wire. This can be implemented by the web heads in front of the CouchDB server on the server side. Such a "/db/doc_diff" method can then do a GET to the database server, get the last version, apply the diff that came over the network, and then do a PUT, and similarly send a diff based on a GET of the current and the version known to the client. This makes most sense as a generic extension to the protocol to let other data types benefit from this. | CouchDB is not an overly wire-compact protocol. This is particularly painful for bookmarks updates since we repeat a lot of data that is unchanged from the last state. Instead of sending the whole document again, we should instead do a jsondiff and send the diff only. The server and the client both have the last _rev and can reconstruct the full version. Once the initial version of this is operational, we should consider a fairly simple addition to the CouchDB protocol to make it more compact on the wire. This can be implemented by the web heads in front of the CouchDB server on the server side. Such a "/db/doc_diff" method can then do a GET to the database server, get the last version, apply the diff that came over the network, and then do a PUT, and similarly send a diff based on a GET of the current and the version known to the client. This makes most sense as a generic extension to the protocol to let other data types benefit from this. | ||
=== SPDY === | |||
We should use SPDY to enable content compression. | |||