Services/Sync/Developer/StorageFormat

Under a user's /storage path, there's a set of well-known records and storage formats used by the Weave Sync add-on. This document describes those records and their formats, e.g., JS-object format of the JSON-string payload.

Weave Basic Object

All records from the server come back as a JSON string that represent a JS-object with some attributes. The whole object is described by the Sync Server API, but for a given client storage version, only a subset may be used.

Changes from v1 -> v2

The client no longer stores and accesses the parentid and predecessorid fields in the WBO and instead stores them encrypted inside the Browser Object. Only bookmarks used these fields.

Payload: Encrypted Data Object

Individual data engines, e.g., bookmarks, encrypt their Browser Objects payloads before packing it into the payload field of a Weave Basic Object. There is additional metadata about the encryption to help decrypt the BrowserObject payload.

Changes from v1 -> v2

At the payload level, there are two additional fields: IV and hmac. The IV is stored per-record instead of on the /crypto/<engine> so that a different IV can be used per record. The hmac can be used to verify that the encrypted payload has not been tampered with.

Once the ciphertext inside the payload is decrypted to access the Browser Object's payload, there are fields in addition to the usual Browser Object fields for that data type: id and deleted. For all encrypted data objects, there is an id field, which can be used to verify that the encrypted data is for the requested object. Instead of treating empty string payloads as "delete" records, the deleted field will be set to true.

Payload: meta/global

The payload of the meta/global record contains general metadata to describe data like versions and syncID.

Version 2

storageVersion int:
syncID string:
engines object:

Changes from v1 -> v2

The storageVersion is now an integer (2) instead of a string, e.g., "1.1", and will only change when the storage format changes. It will no longer automatically changed to the newest client to write the record and instead only change on an incompatible storage format change.

There is an engine object that has engine names as its fields with corresponding value objects with two fields: syncID and version. The syncID for an engine changes when the data for that engine drastically changes, and the client will respond by re-syncing all of its data. The version of an engine acts like the storageVersion and indicates the Browser Object version for the corresponding data.

Version 1

storageVersion string:
syncID string:

Payload: keys/pubkey

Changes from v1 -> v2

No changes.

Payload: keys/privkey

Changes from v1 -> v2

No changes.