View source for Firefox OS/Syncto

= Syncto =

Syncto is a layer to provide the Kinto API for Firefox Sync in order to integrate with Firefox OS
== High level architecture ==

== Why Syncto? ==
We have several use cases where we need to synchronize data on Firefox OS. We have browser/system related data like the one handled by Firefox Sync (history, forms, passwords, bookmarks, tabs, etc), but we also have data from Gaia and 3rd party applications with a very long variety of use cases including synchronizing SMS, call history, accounts configuration for Email and Calendar apps, settings, alarms, images, videos and a long etc.

For browser/system related data we considered using Firefox Sync directly, but that involves implementing the Firefox Sync client on Firefox OS. We did an analysis of the [http://ferjm.github.io/fxos-sync-architecture/1-fxos-sync.html#/ potential architecture] of such a client and we even started working on it, but we stopped that work considering that Syncto looks like a better fit for Firefox OS.

=== Lack of backup reliability ===
Firefox Sync is not a backup solution and it does not mean to be one.

On a smartphone, when you lose your mobile, you expect a backup mechanism that restores your new phone to the state it had when it was lost.

Firefox Sync doesn't guarantee that your data will still be there when you restore your new phone.

This is one of the main reasons why we don't want to use Firefox Sync to sync new Firefox OS data.

Kinto is a storage solution to sync and share data that integrates with Firefox Account.

Firefox OS can use Kinto to backup Firefox OS specific data and Syncto which implements the Kinto protocol to access and upload data to Firefox Sync.

=== Client updates ===
As mentioned above, we studied the possibility of implementing a [http://ferjm.github.io/fxos-sync-architecture/1-fxos-sync.html#/ Firefox Sync client on Firefox OS] and we decided that the best approach could be to reuse some of the parts that were already implemented on the platform (Firefox Accounts client, HAWK client, Token server client, etc). That would save us some development time, but would also tie us to Gecko, which means potentially less frequent updates (which are specially rare on Firefox OS). Having a content side solution (kinto.js) opens the door to easier updates and moving most part of the Sync client logic to the server side means mostly seamless updates. We could potentially even add the Kinto client as a Firefox OS addon and update it from the Marketplace when needed, but that's something that we still need to study in detail since addons are quite a new thing on Firefox OS.

=== Client uniformity ===
As mentioned above as well, we have a lot of use cases for synchronizing data on Firefox OS. But we'd like to avoid implementing a specific solution for each use case whenever it's possible. We should aim to have a standard and uniform way of synchronizing data on FirefoxOS. And Kinto looks like the best candidate to provide that. We were already planning to use Kinto to synchronize in-app data, which is the biggest set of use cases for FirefoxOS, so having the chance to also synchronize browser/system related data with the same API makes a lot of sense.
There is already a [https://github.com/mozilla-services/cliquetis/ Kinto client named kinto.js] and written in JS that empowers synchronization between a server and an local IndexedDB database with offline first support and that's exactly what we need on Firefox OS. Having this client on Firefox OS will allow us to fulfill most part of the use cases we identified so far.

=== Push Kinto and Firefox Accounts ecosystem forward ===
* The idea behind Syncto is to build around the Kinto ecosystem.
* Concentrate the effort on building the Kinto.js library that will be used for other Firefox OS synchronization and sharing needs.
* Demonstrate with a production ready solution that the Kinto protocol can replace Firefox Sync needs and later provide a backup solution for Firefox Browser.
* Help growing the storage solution around Firefox Account
* Help growing the Firefox Account ecosystem and get rid of the old BrowserID protocol for Firefox Account in favour of the OAuth Bearer Token flow.

= Implementation =

Adding Firefox OS to the FxSync family involves various components (nice diagram of this coming soon):
* [https://github.com/mozilla-services/syncto syncto server] - sits inbetween the device and the actual Sync server, as a proxy.
* A new 'synchronizer' background app (initially behind a config flag)
* Adapters for history, passwords, tabs, and bookmarks, which plug into this background app
* Changes to the settings app (initially behind a config flag) to activate and configure Sync
* Changes to the system app (initially behind a config flag) to wake up the synchronizer app and report successful/failed result back to the settings app

== The synchronizer app ==

The synchronizer app uses [https://github.com/Kinto/kinto.js kinto.js] to handle the sync process, together with [https://github.com/michielbdejong/fxsync-webcrypto fxsync-webcrypto] which takes care of the WebCrypto calls.

It has various 'adapters', which handle the communication with the device's various DataStores and other low-level APIs (initially only for history and passwords, but adapters for tabs and bookmarks will probably also follow later). FxDesktop currently stores data for these four collections, plus addons and preferences [?: what about form data?].

= Roadmap =

The work is planned as part of the Data Sync sprints, using the [https://bugzilla.mozilla.org/show_bug.cgi?id=824026 Firefox Sync client for Firefox OS] meta bug.

Open bugs that block it include, in [https://wiki.mozilla.org/Firefox_OS_Data_Sync/Scrum/fxos-s5 sprint 5]:

* [https://bugzilla.mozilla.org/show_bug.cgi?id=1191776 Crypto library for Syncto client]
* [https://bugzilla.mozilla.org/show_bug.cgi?id=1191774 Kinto.js next steps discussion]
* [https://bugzilla.mozilla.org/show_bug.cgi?id=1191773 History data adapter]
* [https://bugzilla.mozilla.org/show_bug.cgi?id=1191771 Syncto documentation]
* [https://bugzilla.mozilla.org/show_bug.cgi?id=1191770 Add a way to enable/disable Sync at build time]
* [https://bugzilla.mozilla.org/show_bug.cgi?id=877648  storing credentials from the visited web pages (Password Manager)]

And in sprint 6 and beyond:

* [https://bugzilla.mozilla.org/show_bug.cgi?id=1183103  Refetch FxA credentials (including sync keys if needed) when "services.sync.enabled" pref changes]
* [https://bugzilla.mozilla.org/show_bug.cgi?id=1168185 Sync UI]
* [https://bugzilla.mozilla.org/show_bug.cgi?id=1168160 Sync state machine]

We'll add more bugs as the work progresses.