Services/Sync/NextGen

	THIS PAGE IS A WORKING DRAFT
	The page may be difficult to navigate, and some information on its subject might be incomplete and/or evolving rapidly. If you have any questions or ideas, please add them as a new topic on the discussion page.

Overview

Sync is a highly useful feature, but has so far failed to achieve widespread adoption. We believe Sync to be an important piece of the Firefox experience, and our goal is to deliver a much more attractive product to support continued Firefox growth. We are tentatively targeting Firefox 20 as the GA for this new version of the service.

Use Cases

Needs a narrative for each

v1

• A Firefox user with multiple devices (desktop or mobile) who wants to have their awesomebar and passwords available on all of their devices.

• A Firefox user setting up a new/replacement device wants to get up and running with their full Firefox experience as fast as possible.

• A Firefox user who wants to back up their data easily and safely, and only has a single device.

• A Firefox user who wants to have the option of secure syncing between devices, using a strong key they manage themselves. (Parity with/comparisons to Chrome Sync, OS X Lion DiskVault).

TBD

• A Firefox user who has lost their device, and their Persona password, and wants to recover their data should be able to recover everything except for passwords (and any future auth data we start syncing, such as client certificates).

• Firefox can link external services to user data on the server side.
  • Hypothetical example would be to connect del.icio.us to a user's bookmarks, and the service would connect to the server store directly, instead of requiring a client plug-in.
  • This would involve storing data in plaintext, and would almost certainly be opt-in for those users who want to use the service, possibly in the same service, or in a special-built service like AITC.

User Data covered by the feature

• Feature and defaults parity with the current Firefox Sync offering, unless there is data to support different choices.
  • Bookmarks and History (assuming that's enough to make my awesomebar go) and Passwords is Products minimum requirement.

High level technical description

Identification

The implementation of the next generation Sync feature will be centered around Persona. The Identity team will work to define and deploy a special type of IdP, which I'll refer to as a "Preferred IdP" or PIdP for short. This IdP will support an auth protocol designed to prevent disclosure of the user's Persona passphrase to the server, as well as enabling non-browser clients to directly authenticate without a browser context. In addition to a secure authentication protocol, a PIdP will also expose an API to obtain a wrapped encryption key (known as the User Key, or UK). This key will be encrypted and decrypted on the client, using a key derived from the Persona passphrase. The User Key will not be exposed to consumers, but will be used by the Persona client to encrypt and decrypt a per-service encryption key (known as a Service Key or SK) on request.

Service Authentication

As with Apps in the Cloud, the service client will obtain an assertion from the Persona client, and use that to authenticate to the Token Server. A successful request to the Token Server will return an object containing all necessary data to make requests against the Identity-attached service, as described in the Token Server API docs.

Using the Service

The new version of the Sync Storage API uses the same request authentication model as Apps in the Cloud. Beyond that, it is a significantly refined version of the 1.1 API with a set of improvements outlined in the API docs. Sync clients will use a Service Key stored on the service itself, encrypted by the Persona client using the UK. The SK (or keys chained to it) will be used to encrypt outgoing records and decrypt incoming records, like the existing model.

Open Questions

Product Questions

• Are we still planning to ship old and new Sync products in parallel, or are we doing migration on upgrade? (Asa)
  • mconnor: This has a non-zero cost, and we're talking about 0.5% of Firefox users. We can do it, but it adds difficult-to-scope time/complexity.
  • For how long?
  • Assumption: even if shipping in parallel, not simultaneously functional. This would be an undue development and testing burden.
    • We're going to ship in parallel until we reduce the users of previous generation sync to some minimal level at which point we'll EOL the previous version. - A

• J-PAKE as an option for users? (Asa)
  • mconnor: Not required for sure, but if we're allowing users to provide their own keys, manual entry really sucks on mobile. NFC could be a future option, making this unnecessary, but we should decide this explicitly. The cost to maintain/preserve is not dramatically high, but is of course more than zero.
    • I don't think we should do this. I know typing on some mobile devices is a pain but I don't think the J-PAKE solution conceptually less burden. - A

• Is recovery after password reset a required v1 use-case? (Asa, Karen, mconnor)
  • mconnor: Timeline/model for an escrow service is as yet undefined, adding substantial schedule risk. My recommendation is that this is a nice-to-have for v1 of the new service.
    • We don't have any recovery system now, right? If that's the case, I think we could make recovery a high priority v2 feature. We should talk about this because I don't know what's involved. - A

• What degree of durability is required for data stored on the service? (Asa, mconnor)
  • mconnor: This is going to be a budget question. It needs a cost/benefit tradeoff from mmayo's team. From prior discussions, more than 99.9999999% will be much more expensive, but 1 in a billion feels pretty okay.
  • Cost estimates and recommendations coming from Bob soonish.
    • I don't know what industry standard is. Does anyone else? I also don't know how to think about durability because at a certain point the numbers get too small/big for my little brain to process intelligently. - A

• What is the user expectation for data availability, above and beyond "the bits in the cloud don't go away"? Durability would mean we don't lose your bookmarks if you do nothing, but beyond that users might want to get back their bookmarks after they accidentally deleted them all. To what extent should backup/versioning etc. feature in our roadmap?
  • • I hadn't considered versioning so this is just 'off the top of my head'. If we could do a "dated profile backup" every so often, plus before a Firefox upgrade, like a Windows Restore point, that would be pretty swell. - A

• Is there a per-user uptime requirement? (Asa, mconnor)
  • mconnor: I would propose 99.9% for any given user's data access, at the high end. This is the high end of what is viable with a single datacenter, and multi-datacenter would be prohibitively expensive.
  • How would you define uptime requirement? A see many options. At any time N% of users have Sync working for them. Sync is working for any user N% of the time. Sync is available to a particular user N% of the time they attempt to connect to it. etc.
    • Can we separate out "uptime" from "times when the service needs to be up"? I mean, for example, if it's always up when I'm setting up a new device but it's somewhat less consistently up when I'm just making incremental updates? As we add "instant" items to this feature, like "send this tab to my phone now" we sure don't want to have a "sorry, couldn't connect, please wait for the service to return" message or anything like that. I'd also ask "what's industry standard" for things like this -- a high-profile consumer service. - A

• How should migration be messaged? Is it sufficient to have a note on the newly set-up device saying that your other devices need to be upgraded? A non-fatal notice on the old devices (requires work on Sync 1.1)? A fatal error on the old devices?
  • • That sounds pretty good. Could we somehow kick the updater in the other clients if we see that one client has moved forward a version, either triggering the update or prompting for a restart if the update's already staged? - A

• To what extent will we be pushing this feature? Privileged account + durable storage implies backup, and if half a billion users sign up, we need to reflect that in capacity planning! This is very relevant to mmayo's team, and affects the operational timeline.
  • • From a Product perspective, I want 100% of Firefox users using this service. I don't expect to reach that level of usage any time soon, but I'd say 10x what we currently have is the kind of boost I'm looking for in the short term. - A

• Do we want the PIdP to be featured as browser-wide UX, or only as part of Sync? (Asa, Karen)
  • This requires thought for technical delivery on mobile, too.
    • I'm not sure I understand this question. Tell me if this answer fits that question. We want users to create a Firefox account. If when they're creating that account, we see they're entering an existing Persona ID, we should prompt them to "upgrade" that ID (adding the Firefox Sync/Backup capability to that Persona ID) by re-typing their Persona password. Similarly, if at a later time the user encounters a site that utilizes Persona ID for logins, and the user starts typing in their Firefox ID, we can offer to "upgrade" their Firefox account to a Persona ID credential. On the backend they're the same thing just with one bit set for Firefox enabled and one bit set for Persona ID enabled. Thunder has this all sorted out -- assuming I'm answering the question you've asked. - A

Technical Questions

• Is the key storage (wrapped master key) service a part of the Preferred Idp API, or is it an Identity Attached Service? (Ben, mconnor)

• What is the timeline for delivering key-wrapping as a capability? (Ben, bwarner)

• Are any further changes required to the 2.0 protocol? (gps, rnewman, rfkelly)

• Are there any further changes required to the v6 storage format? (gps)

• How does the account/PIdP login work on the client -- is this a general feature for the browser, or a Sync-specific feature?

More details are in the technical plan.