Services/Sync/Protocol 2.0

Please use "Edit with form" above to edit this page.

Status

Sync Storage Protocol 2.0
Stage	Draft
Status	`
Release target	`
Health	OK
Status note	`

{{#set:Feature name=Sync Storage Protocol 2.0

|Feature stage=Draft |Feature status=` |Feature version=` |Feature health=OK |Feature status note=` }}

Team

Product manager	`
Directly Responsible Individual	`
Lead engineer	`
Security lead	`
Privacy lead	`
Localization lead	`
Accessibility lead	`
QA lead	`
UX lead	`
Product marketing lead	`
Operations lead	`
Additional members	`

{{#set:Feature product manager=`

|Feature feature manager=` |Feature lead engineer=` |Feature security lead=` |Feature privacy lead=` |Feature localization lead=` |Feature accessibility lead=` |Feature qa lead=` |Feature ux lead=` |Feature product marketing lead=` |Feature operations lead=` |Feature additional members=` }}

Open issues/risks

`

Stage 1: Definition

1. Feature overview

A new Sync HTTP storage protocol will be defined and implemented. The new protocol will address known problems with the existing protocol specification, remove old features, modify behavior, and add new APIs.

2. Users & use cases

`

3. Dependencies

`

4. Requirements

`

Non-goals

`

Stage 2: Design

5. Functional specification

Random wishlist items:

• Sessions. The client has a concept of beginning and ending a sync, with some expectation of consistency within a session -- for example, that nobody else will be racing to write to meta/global while we upload items. Client behavior could be simplified (e.g., eliminating race-reduction handling) and made more robust if sessions were explicit.
  • Related - moving away from millisecond timestamp precision

• Cross-collection operations.

• URL space separations. Consider whether we should re-partition the URL space to allow for operations such as "delete everything apart from meta/global", "wipe all collections apart from keys", or "kill all dependent storage".

• Ability to send one-time messages to clients (such as "this version is being deprecated. Please upgrade")

• Additional information included with X-Weave-Backoff

• Rate-limiting clients who send too-large POST and PUT bodies. 413 with a header.

• If we don't already do it, we should take advantage of more opportunities to send less data: for example, sending 304 Not Modified in response to info and meta requests. In a world of instant syncs, this might avoid schlepping a non-trivial number of bytes over the wire, and some client parsing too.

• Ability to tell client that it isn't supported and needs to upgrade to a new version or switch to a new version of the protocol.

The following API additions are proposed:

The following API removals are proposed:

• Remove use of HTTP 400. While 400 has been overloaded by many, there are still some practical considerations for not using it. Some HTTP stacks and agents (e.g. load balancers) will kill the underlying connection if a 400 is seen. Error handling should be accomplished some other way.

The following API clarifications are needed:

• Define relationship between X-Weave-Backoff and Retry-After HTTP response headers.

See also: Labs/Weave/Sync/1.1/2.0changes

6. User experience design

`

Stage 3: Planning

7. Implementation plan

`

8. Reviews

Security review

`

Privacy review

`

Localization review

`

Accessibility

`

Quality Assurance review

`

Operations review

`

Stage 4: Development

9. Implementation

`

Stage 5: Release

10. Landing criteria

` {{#set:Feature open issues and risks=` |Feature overview=A new Sync HTTP storage protocol will be defined and implemented. The new protocol will address known problems with the existing protocol specification, remove old features, modify behavior, and add new APIs. |Feature users and use cases=` |Feature dependencies=` |Feature requirements=` |Feature non-goals=` |Feature functional spec=Random wishlist items:

• Sessions. The client has a concept of beginning and ending a sync, with some expectation of consistency within a session -- for example, that nobody else will be racing to write to meta/global while we upload items. Client behavior could be simplified (e.g., eliminating race-reduction handling) and made more robust if sessions were explicit.
  • Related - moving away from millisecond timestamp precision

• Cross-collection operations.

• URL space separations. Consider whether we should re-partition the URL space to allow for operations such as "delete everything apart from meta/global", "wipe all collections apart from keys", or "kill all dependent storage".

• Ability to send one-time messages to clients (such as "this version is being deprecated. Please upgrade")

• Additional information included with X-Weave-Backoff

• Rate-limiting clients who send too-large POST and PUT bodies. 413 with a header.

• If we don't already do it, we should take advantage of more opportunities to send less data: for example, sending 304 Not Modified in response to info and meta requests. In a world of instant syncs, this might avoid schlepping a non-trivial number of bytes over the wire, and some client parsing too.

• Ability to tell client that it isn't supported and needs to upgrade to a new version or switch to a new version of the protocol.

The following API additions are proposed:

The following API removals are proposed:

• Remove use of HTTP 400. While 400 has been overloaded by many, there are still some practical considerations for not using it. Some HTTP stacks and agents (e.g. load balancers) will kill the underlying connection if a 400 is seen. Error handling should be accomplished some other way.

The following API clarifications are needed:

• Define relationship between X-Weave-Backoff and Retry-After HTTP response headers.

See also: Labs/Weave/Sync/1.1/2.0changes |Feature ux design=` |Feature implementation plan=` |Feature security review=` |Feature privacy review=` |Feature localization review=` |Feature accessibility review=` |Feature qa review=` |Feature operations review=` |Feature implementation notes=` |Feature landing criteria=` }}

Feature details

Priority	Unprioritized
Rank	999
Theme / Goal	`
Roadmap	`
Secondary roadmap	`
Feature list	`
Project	`
Engineering team	`

{{#set:Feature priority=Unprioritized

|Feature rank=999 |Feature theme=` |Feature roadmap=` |Feature secondary roadmap=` |Feature list=` |Feature project=` |Feature engineering team=` }}

Team status notes

	status	notes
Products	`	`
Engineering	`	`
Security	`	`
Privacy	`	`
Localization	`	`
Accessibility	`	`
Quality assurance	`	`
User experience	`	`
Product marketing	`	`
Operations	`	`

{{#set:Feature products status=`

|Feature products notes=` |Feature engineering status=` |Feature engineering notes=` |Feature security status=` |Feature security health=` |Feature security notes=` |Feature privacy status=` |Feature privacy notes=` |Feature localization status=` |Feature localization notes=` |Feature accessibility status=` |Feature accessibility notes=` |Feature qa status=` |Feature qa notes=` |Feature ux status=` |Feature ux notes=` |Feature product marketing status=` |Feature product marketing notes=` |Feature operations status=` |Feature operations notes=` }}