canmove, Confirmed users
640
edits
Services/Sync/Developer/StorageFormat: Difference between revisions
Services/Sync/Developer/StorageFormat (view source)
Revision as of 00:40, 11 August 2011
, 11 August 2011→Version 5
m (→Version 5) |
|||
| (6 intermediate revisions by one other user not shown) | |||
| Line 7: | Line 7: | ||
All records from the server come back as a JSON string that represent a JS-object with some attributes. The whole object is described by the [[Labs/Weave/API|Sync Server API]], but for a given client storage version, only a subset may be used. | All records from the server come back as a JSON string that represent a JS-object with some attributes. The whole object is described by the [[Labs/Weave/API|Sync Server API]], but for a given client storage version, only a subset may be used. | ||
== Versions 2, 3, and | == Versions 2, 3, and 5 == | ||
The following describes the JS-object represented by the JSON-string record: | The following describes the JS-object represented by the JSON-string record: | ||
| Line 15: | Line 15: | ||
! id | ! id | ||
| string | | string | ||
| | | Record identifier. Starting with version 5 this SHOULD be exactly 12 characters from the base64url alphabet. | ||
|- | |- | ||
! modified | ! modified | ||
| Line 84: | Line 84: | ||
Individual data engines, e.g., bookmarks, encrypt their [[Labs/Weave/Developer/BrowserObjects|Browser Objects]] payloads before packing it into the <tt>payload</tt> field of a Weave Basic Object. There is additional metadata about the encryption to help decrypt the BrowserObject payload. | Individual data engines, e.g., bookmarks, encrypt their [[Labs/Weave/Developer/BrowserObjects|Browser Objects]] payloads before packing it into the <tt>payload</tt> field of a Weave Basic Object. There is additional metadata about the encryption to help decrypt the BrowserObject payload. | ||
== Version | == Version 5 == | ||
Version 5 is similar to Version 2 and 3, except: | |||
* the '''encryption''' field (which specifies the key to use) has been dropped; key selection is now implied by the WBO's collection | |||
* the HMAC is now calculated with a separate key, used in its raw byte form rather than in base64 encoding. | |||
The keys with which to verify and decrypt a WBO are now determined based on the collection name. If collection-specific keys do not exist, the default key bundle is used. | |||
{| cellpadding=5 | |||
! ciphertext | |||
| string | |||
| Encrypted JSON-stringified Browser Object | |||
|- | |||
! IV | |||
| string | |||
| Initialization vector used when decrypting the <tt>ciphertext</tt> | |||
|- | |||
! hmac | |||
| string | |||
| SHA256 HMAC in hex representation, computed on the base64 encoded version of the ciphertext, using the byte-representation of the HMAC key from the same bundle as the encryption key. | |||
|} | |||
=== Example === | === Example === | ||
| Line 93: | Line 113: | ||
"IV":"GluQHjEH65G0gPk/d/OGmg==", | "IV":"GluQHjEH65G0gPk/d/OGmg==", | ||
"hmac":"c550f20a784cab566f8b2223e546c3abbd52e2709e74e4e9902faad8611aa289"}</pre> | "hmac":"c550f20a784cab566f8b2223e546c3abbd52e2709e74e4e9902faad8611aa289"}</pre> | ||
== Version 3 == | == Version 3 == | ||
| Line 125: | Line 144: | ||
! hmac | ! hmac | ||
| string | | string | ||
| SHA256 HMAC computed on the base64 encoded version of the ciphertext, using a base64 encoded version of the key used to encrypt the ciphertext in the first place as the key to the hmac algorithm | | SHA256 HMAC in hex format, computed on the base64 encoded version of the ciphertext, using a base64 encoded version of the key used to encrypt the ciphertext in the first place as the key to the hmac algorithm. | ||
|} | |} | ||
| Line 176: | Line 195: | ||
The payload of the <tt>meta/global</tt> record contains general metadata to describe data like versions and syncID. | The payload of the <tt>meta/global</tt> record contains general metadata to describe data like versions and syncID. | ||
== Version 2, 3 and | == Version 2, 3 and 5 == | ||
The following describes the JS-object represented by the JSON-string payload: | The following describes the JS-object represented by the JSON-string payload: | ||
| Line 232: | Line 251: | ||
= Payload: crypto/keys = | = Payload: crypto/keys = | ||
In storage | In storage Version 5, the public/private key layer has been dropped. All bulk keys are now stored in this one WBO. Encryption and HMAC keys are separate keys and kept in key pairs. | ||
== Version | The '''keys''' WBO is encrypted and verified just like any other WBO, except a different key bundle is used. The key bundle for the '''keys''' WBO is derived from the Sync Key using an HKDF with HMAC-SHA256 as the HMAC function (see [http://tools.ietf.org/html/rfc5869 RFC 5869]): | ||
Pseudo-code: | |||
HMAC_INPUT = "Sync-AES_256_CBC-HMAC256" | |||
encryption_key = HMAC-SHA256(sync_key, "" + HMAC_INPUT + username + "\x01") | |||
hmac_key = HMAC-SHA256(sync_key, encryption_key + HMAC_INPUT + username + "\x02") | |||
Here '''sync_key''' is the 16 byte representation of the Sync Key. To translate between the byte and user-readable translation, base32 is used, although with a slightly different alphabet than what [http://tools.ietf.org/html/rfc4648 RFC 4648] uses. For readability reasons, 'l' has been replaced with '8' and 'o' with '9': | |||
sync_key = decodeBase32(sync_key_ui.replace('8', 'l').replace('9', 'o')) | |||
sync_key_ui = encodeBase32(sync_key).replace('l', '8').replace('o', '8) | |||
== Version 5 == | |||
{| cellpadding=5 | {| cellpadding=5 | ||
| Line 249: | Line 282: | ||
| Currently defaulting to "crypto" | | Currently defaulting to "crypto" | ||
|} | |} | ||
=== Example === | |||
{"id":"keys", | |||
"collection":"crypto", | |||
"collections":{}, | |||
"default:['dGhlc2UtYXJlLWV4YWN0bHktMzItY2hhcmFjdGVycy4=', | |||
'eWV0LWFub3RoZXItc2V0LW9mLTMyLWNoYXJhY3RlcnM=']} | |||
= Payload: keys/pubkey = | = Payload: keys/pubkey = | ||