MozillaWiki

Firefox OS Data Sync: Difference between revisions

Page Discussion

Firefox OS Data Sync (view source)

Revision as of 08:21, 1 February 2016

8,765 bytes added ,  1 February 2016
→‎2.6
 
(30 intermediate revisions by 5 users not shown)
Line 18: Line 18:


== Use cases ==
== Use cases ==
=== Messaging application ===
=== Browser-related data ===
Alice uses her Messaging app to send and receive SMS, MMS and IM. She accesses this application from her Firefox OS tablet, her desktop browser and her Android phone. She can see and manage the history and content of the messages sent and received from any of these devices. She can continue writing an IM that she started typing on her Android phone on her desktop browser app.
==== Saved passwords ====
Alice has been using FxDesktop for a while, and has saved the passwords for her favorite websites in FxDesktop Saved Passwords. She is already using FxSync on FxDesktop. Once she enables FxSync on her FxOS phone, the next time she visits one of these websites with her phone, the password to log in is prefilled with the one from FxSync. From now on, if she updates or adds a password on any device, this will also be updated on her other devices.
 
==== Browser History ====
Alice has been using FxDesktop for a while, and is already using FxSync there. Once she enables FxSync on her FxOS phone, the next time she opens the browser, she will see items from her FxDesktop history show up in the thumbnail tiles. Also, when she starts typing a URL, it autocompletes from the history that was synced from FxDesktop. From now on, if she visits a website on any device, this will also be updated in the browser history on her other devices.
 
==== Bookmarks ====
Alice has been using FxDesktop for a while, and has collected a nice set of bookmarks for her favorite websites there. She is already using FxSync on FxDesktop. Once she enables FxSync on her FxOS TV, the next time she browses the web with her TV, her bookmarks from FxSync are available. From now on, if she adds or updates a bookmark on any device that supports bookmarks, this will also be updated on her other devices.
 
=== Backup/Restore device ===
Alice purchases a new Firefox OS phone (\o/). She already owns a Firefox OS tablet and she wants to have the same experience and data in both devices. She enters her Firefox Accounts credentials while configuring her new device. Her new device installs all the applications that she has on her tablet, the homescreen wallpaper, the passcode for the lockscreen, the notification sounds. When she opens the Gallery app in her new device, she is asked if she wants to access her photo collection from her new device.
 
=== Backup/Sync contacts ===
So far, Firefox OS support one-shot import contacts from Gmail or from Hotmail/Outlook. We want to augment this functionality to allow one-shot import from any CardDAV server, as well as repeated imports (staying in sync), and export.


=== Media files ===
=== Backup/Sync media files ===
Bob uses his Music app to listen to music and audio files. He keeps a library with his preferred titles. He adds new songs from his desktop browser. When he uses his Firefox OS device, he can listen to these new songs if he is online. He can also choose to download them so he can play them offline.
Bob uses his Music app to listen to music and audio files. He keeps a library with his preferred titles. He adds new songs from his desktop browser. When he uses his Firefox OS device, he can listen to these new songs if he is online. He can also choose to download them so he can play them offline.


''Question: Does the [[https://wiki.mozilla.org/Gaia/Architecture_Proposal|proposed V3 architecture]] buy us anything in terms improved offline experience here? -Stephany''
''Question: Does the [[Gaia/Architecture_Proposal|proposed V3 architecture]] buy us anything in terms improved offline experience here? -Stephany''


Alice listens to her favorite knitting podcasts -- Knitting Pipeline and The Knit Girllls -- in her web browser (while knitting beside her laptop during business travel) or via the Apple TV > Podcasts section, when she is at home. Alice can access both podcasts on the Web and in iTunes. She has a Lenovo PC so doesn't really use iTunes outside of Apple TV. When Alice sits down in her plane seat for her next business trip, plugs in her headphones and clicks the play button on the video clip embedded in the podcast's web page, Firefox Cloud asks if she'd like the video to be backed up so she can pick it up anywhere (on her phone, in case her laptop battery dies; on her laptop when it's offline during the flight with alleged but nonexistent Internet service), where she left off.
Alice listens to her favorite knitting podcasts -- Knitting Pipeline and The Knit Girllls -- in her web browser (while knitting beside her laptop during business travel) or via the Apple TV > Podcasts section, when she is at home. Alice can access both podcasts on the Web and in iTunes. She has a Lenovo PC so doesn't really use iTunes outside of Apple TV. When Alice sits down in her plane seat for her next business trip, plugs in her headphones and clicks the play button on the video clip embedded in the podcast's web page, Firefox Cloud asks if she'd like the video to be backed up so she can pick it up anywhere (on her phone, in case her laptop battery dies; on her laptop when it's offline during the flight with alleged but nonexistent Internet service), where she left off.


=== Backup ===
=== Messaging application ===
Alice purchases a new Firefox OS phone (\o/). She already owns a Firefox OS tablet and she wants to have the same experience and data in both devices. She enters her Firefox Accounts credentials while configuring her new device. Her new device installs all the applications that she has on her tablet, the homescreen wallpaper, the passcode for the lockscreen, the notification sounds. When she opens the Gallery app in her new device, she is asked if she wants to access her photo collection from her new device.
Alice uses her Messaging app to send and receive SMS, MMS and IM. She accesses this application from her Firefox OS tablet, her desktop browser and her Android phone. She can see and manage the history and content of the messages sent and received from any of these devices. She can continue writing an IM that she started typing on her Android phone on her desktop browser app.


=== File sharing ===
=== File sharing ===
Line 43: Line 56:


=== Other use cases ===
=== Other use cases ===
You can see the result of a previous discussion around data sync use cases [https://wiki.mozilla.org/Firefox_OS_Data_Sync_Use_Cases_Discussion here].
You can see the result of a previous discussion around data sync use cases [[Firefox OS Data Sync Use Cases Discussion|here]].


== Data types ==
== Data types ==
Looking at the proposed use cases and the nature of the data to be synchronized we can group it in:
Looking at the proposed use cases and the nature of the data to be synchronized we can group it in:
=== Browser/System data ===
=== Browser/System data ===
This includes things like browsing history, bookmarks, passwords, form autocomplete data, list of installed apps, system settings, etc.
This includes things like browsing history, bookmarks, passwords, form autocomplete data, list of installed apps, system settings, etc.
=== In-app generated data ===
=== In-app generated data ===
This includes all the data that is not owned by the system. And can be also grouped in:
This includes all the data that is not owned by the system. And can be also grouped in:
Line 55: Line 70:
* '''Documents'''
* '''Documents'''
Everything that fits on an indexedDB like database. Including in-app specific settings or things like SMS, call history, alarms, etc.
Everything that fits on an indexedDB like database. Including in-app specific settings or things like SMS, call history, alarms, etc.
== Projects ==
The Firefox OS Data Sync team will be working on several projects during 2016.
See [https://wiki.mozilla.org/Firefox_OS_Cloud_Roadmap Firefox OS Cloud Roadmap] for an extensive list of product ideas.
=== Firefox Sync ===
We have implemented import-only bookmarks and history synchronization
for the Smart TV in FxOS 2.5. We will be enabling FxSync on master for phone foxfooders soon.
The next logical step is to finish the work that we started with Firefox Sync
in a way that Firefox OS can catch up with other Firefox Sync clients (Desktop,
Android and iOS). So we need to work on exporting data from the collections that
we already have implemented (bookmarks and history) and work on importing/exporting
from the collections that it still doesn't have (passwords, add-ons, tabs,
preferences, forms, clients). Some of these may not make sense on Firefox OS though.
Also for bookmarks, we still need to define and implement an UI to display the
synchronized bookmarks on the phone.
Finally, we need to allow the creation of new Sync users from Firefox OS. This
involves generating the 'crypto/keys' object containing a new bulk key bundle,
encrypt it with 'kB' on the client and uploading it to the server.
* Stage 1 (completed): Read-only history and bookmarks sync on the Smart TV.
* Stage 2 (completed): Read-only history sync on the phone.
* Stage 3 (to be discussed, [https://groups.google.com/forum/#!topic/mozilla.dev.fxos.sync/DnL1WQ5L24I see ML]): Read-write history sync and passwords sync on the phone.
* Stage 4 (to be discussed, [https://groups.google.com/forum/#!topic/mozilla.dev.fxos.sync/DnL1WQ5L24I see ML]): Sync the [https://docs.services.mozilla.com/sync/objectformats.html rest of the collections] as and when the phone starts supporting these features.
See also https://wiki.mozilla.org/Firefox_OS/Syncto.
=== Media Files Sync ===
Planned for 2016. FxOS already allows users to store and view media files using storage that is connected to the device (for instance, an SD-card). Mounting remote cloud storage space (for instance the user's Dropbox or ownCloud account) as a (cached) virtual storage device makes it possible to browse the files on there 'as if' they were on a local storage device.
=== Contacts Sync ===
See [https://groups.google.com/forum/#!topic/mozilla.dev.fxos.sync/ctlFp_2_nlA this mailing list thread] for work going on during 2015-Q4. During 2016, we would like to improve the existing sync (currently more a one-shot import) with Gmail and Outlook contacts, and expand it to allow sync with any CardDAV server.
=== Backup / Restore ===
Planned for 2016. It should be possible to take snapshots of the entire data state of a Firefox OS device, and later restore that snapshot onto the same or a new device. During Mozlando we will discuss whether this can be a pure peer-to-peer architecture (e.g. using RTCPeerConnection), or whether it will be necessary to store these snapshots on a server where the user has storage space. See https://wiki.mozilla.org/Firefox_OS_Cloud_Backup_Restore for an extensive list of product requirements.
=== Data Sync for the Web ===
Apart from launching specific sync features on upcoming Firefox OS versions, we will aim to discuss, build, and standardize the general architecture of data sync technologies that are currently missing on the web platform, but are needed to make data sync on the (mobile) web as easy to use as the data sync features of proprietary single-vendor platforms.


= The solution(s) =
= The solution(s) =
== For Browser/System data ==
== For Browser/System data ==
For '''Browser/System''' specific data we chose to use the existing [[Services/Sync|Firefox Sync]] platform. That will give users a way to backup Firefox OS system data like the list of installed apps, the browser history, their bookmarks, etc. and to access this data from any Firefox product (Desktop, Android, iOS, OS).
For '''Browser/System''' specific data we chose to use the existing [[Services/Sync|Firefox Sync]] platform. That will give users a way to backup some of the browser-related system data like saved passwords, the browser history, their bookmarks, etc. and to access this data from any Firefox product (Desktop, Android, iOS, OS).
It seems that we will probably need to rewrite a lot of the code that currently exists in Gecko and we will need to expose a Gaia only API to allow apps to feed the platform with this kind of data from the content side.
We will be adding more details once we figure out all the required work.


== For in-app generated data ==
== For in-app generated data ==
We are still not sure about which approach we should take to allow the apps to store and synchronize '''in-app generated data''' with a remote storage. We know some requirements we would like to address though.
We are still not sure about which approach we should take to allow the apps to store and synchronize '''in-app generated data''' with a remote storage. We know some requirements we would like to address though.
=== Requirements for in-app data sync ===
=== Requirements for in-app data sync ===
* '''Authentication'''
* '''Authentication'''
Line 94: Line 149:


The code and documentation for this generic storage can be found at:
The code and documentation for this generic storage can be found at:
 
* [http://www.servicedenuages.fr/en/generic-storage-ecosystem Blog post]
* https://github.com/mozilla-services/cliquet
* https://github.com/mozilla-services/cliquet
* https://github.com/mozilla-services/kinto
* https://github.com/mozilla-services/kinto
Line 109: Line 164:
* Outlook for Contacts. We are currently importing contacts only.
* Outlook for Contacts. We are currently importing contacts only.
* CardDav for Contacts. One of the most wanted features (269 votes so far). Check [https://bugzilla.mozilla.org/show_bug.cgi?id=859306 bug 859306].
* CardDav for Contacts. One of the most wanted features (269 votes so far). Check [https://bugzilla.mozilla.org/show_bug.cgi?id=859306 bug 859306].
= Ideas for Architecture Improvements =
== SyncEngine shared library ==
Right now, the sync engine lives only inside the Sync app that we added for synchronizing bookmarks and history. This works fine as long as the data source to be synchronized is accessible outside of the owner app (via DataStore for example). But it makes it impossible to synchronize an app's data that is not exposed outside of the owner app. Also, having the Sync app managing the synchronization of any kind of data doesn't really scale very well and causes potentially unnecessary data duplication. It makes more sense that each app handles the synchronization of its own data. So in order to allow the addition of new collections (not only existing Firefox Sync collections, but also any other type of collection like contacts, call log or settings data) from any application we need to package the logic of the sync engine in the form of a shared library that can be used by any Gaia application.
== Sync collection extension mechanism ==
Apart from creating this shared library to ease the synchronization process from other apps, we also want to create a way for apps to register themselves in the Firefox Sync machinery. This is, to be able to add their collections to be listed within the Settings UI and to register themselves to be waken up periodically by the sync scheduler that lives in the System app.
== Push notifications in Kinto ==
Because we want to avoid unnecessary battery and network consumption by polling constantly from the device but we still want the data to be synchronized as soon as possible, we want to add push notifications to Kinto. This way, when something changes in a remote collection, the server will send every client of this collection a notification about this change so it can update itself accordingly.
== Firefox Accounts improvements on Firefox OS ==
This is not entirely part of sync, but it is quite related. We are using Firefox Accounts as our identity mechanism. Firefox OS the only platform that still uses BrowserId assertions while other platforms are using the OAuth flow instead. This forces Relying Parties that want to benefit from the SSO system that we have on FxOS to use the non-standard mozId API. It also forces every server side part that we want to create for syncing to support both BrowserId assertions (if we want to sync from FxOS) and OAuth tokens (if we want to sync from Desktop). We aim to move to OAuth as soon as possible.


= Related work =
= Related work =
Line 115: Line 183:
* [http://dexie.org/ Dexie.js]
* [http://dexie.org/ Dexie.js]
* [https://github.com/remotestorage/remotestorage.js RemoteStorage]
* [https://github.com/remotestorage/remotestorage.js RemoteStorage]
* [https://github.com/michielbdejong/ludbud Lubdub]
* [https://github.com/kurko/ember-sync Ember-sync]
* [https://github.com/kurko/ember-sync Ember-sync]
* [http://hood.ie/ Hoodie]
* [http://hood.ie/ Hoodie]
Line 121: Line 190:


= Toolkit team =
= Toolkit team =
== Mailing list ==
[https://lists.mozilla.org/listinfo/dev-fxos-sync mozilla.dev.fxos.sync]
== Bugs ==
== Bugs ==
https://bugzilla.mozilla.org/show_bug.cgi?id=1161657 Meta bug
<bugzilla>
    {
        "blocks":"1161657",
        "include_fields": "id, summary, component, priority, resolution, assigned_to, depends_on, blocks, whiteboard, cf_blocking_b2g, cf_feature_b2g, target_milestone"
    }
</bugzilla>
 
== Sprints ==
=== 2.5 ===
* [[Firefox OS Data Sync/Scrum/fxos-s4|FxOS-S4]]
* [[Firefox OS Data Sync/Scrum/fxos-s5|FxOS-S5]]
* [[Firefox OS Data Sync/Scrum/fxos-s6|FxOS-S6]]
* [[Firefox OS Data Sync/Scrum/fxos-s7|FxOS-S7]]
* [[Firefox OS Data Sync/Scrum/fxos-s8|FxOS-S8]]
* [[Firefox OS Data Sync/Scrum/fxos-s9|FxOS-S9]]
* [[Firefox OS Data Sync/Scrum/fxos-s10|FxOS-S10]]
* [[Firefox OS Data Sync/Scrum/fxos-s11|FxOS-S11]]
=== 2.6 ===
* [[Firefox OS Data Sync/Scrum/fxos2.6-s2|fxos2.6-s2]]
* [[Firefox OS Data Sync/Scrum/fxos2.6-s3|fxos2.6-s3]]
* [[Firefox OS Data Sync/Scrum/fxos2.6-s4|fxos2.6-s4]]
* [[Firefox OS Data Sync/Scrum/fxos2.6-s5|fxos2.6-s5]]
* [[Firefox OS Data Sync/Scrum/fxos2.6-s6|fxos2.6-s6]]
* [[Firefox OS Data Sync/Scrum/fxos2.6-s7|fxos2.6-s7]]


== Meetings ==
== Stand-up Meetings ==
=== When ===
=== When ===
* Tuesdays and Thursdays at 10am CEST / 16pm CST
=== Where ===
=== Where ===
* ferjm's Vidyo room
* Public link: https://v.mozilla.com/flex.html?roomdirect.html&key=8SacsQs9fUmW
=== What ===
=== What ===
* [https://public.etherpad-mozilla.org/p/data-sync-stand-up Stand-up Etherpad]
Oteo
Confirmed users
1,225

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/1072345...1115174"