Firefox/Kinto

Key features

• Diff-based data synchronization
• Data integrity/signing
• Offline persistence
• Admin panel UI

Use Cases

• Certificates blocklist (OneCRL) (contact: mgoodwin)
• Addons/Plugins/Gfx Blocklisting (contact: leplatrem)
• storage.sync API (WebExtensions) (contact: tarek)
• Fennec assets catalog (contact: Sebastian Kaspari)
• Password manager recipes (contact: MattN)

Feel free to come and discuss on #storage :)

Usage

Leveraging the Kinto HTTP client in Gecko looks like this:

const { KintoHttpClient } = Cu.import("resource://services-common/kinto-http-client.js");

const client = new KintoHttpClient("https://kinto.dev.mozaws.net/v1");
client.bucket("a-bucket").collection("a-collection")
  .listRecords()
  .then(result => ...);

As for the Kinto offline-first client, it is like:

const { loadKinto } = Cu.import("resource://services-common/kinto-offline-client.js");

const KintoOfflineClient = loadKinto();

const client = new KintoOfflineClient({
  adapter: Kinto.adapters.FirefoxAdapter,
  remote: "https://kinto.dev.mozaws.net/v1",
  bucket: "a-bucket"
});

const collection = db.collection("a-collection");

try {
  yield collection.db.open();
  // Fetch changes from server.
  yield collection.sync();
  // Read local collection of records.
  const records = yield collection.list();
} finally {
  yield collection.db.close();
}

Specifications

Currently, the instance of Kinto used by Firefox clients is hosted at https://firefox.settings.services.mozilla.com/v1/

Blocklists

The goal is to replace the current system based on a single XML file downloaded everyday by several Kinto collections.

• The bucket is blocklists
• The collection for OneCRL entries is certificates https://firefox.settings.services.mozilla.com/v1/buckets/blocklists/collections/certificates/records
• The collection for Add-ons entries is addons https://firefox.settings.services.mozilla.com/v1/buckets/blocklists/collections/certificates/records
• The collection for Plugins entries is plugins https://firefox.settings.services.mozilla.com/v1/buckets/blocklists/collections/certificates/records
• The collection for Gfx entries is gfx https://firefox.settings.services.mozilla.com/v1/buckets/blocklists/collections/certificates/records

See https://bugzilla.mozilla.org/show_bug.cgi?id=1197707

Fennec assets catalog

The goal is to remove the static assets (fonts, hyphenation dicts, etc.) from the distribution package and download them asynchronously using an online Kinto catalog.

• The bucket is fennec
• The collection is catalog

See https://bugzilla.mozilla.org/show_bug.cgi?id=1201059

Contribute

Upgrade client libraries

Two client libraries are embedded in Firefox:

• Kinto/kinto-client.js: for direct interactions with the Kinto HTTP API
• Kinto/kinto.js: for offline persistence in internal SQLite

Generate bundles

The Kinto client libraries are developed independently on Github:

• kinto-client is the HTTP client for the Kinto REST API;
• kinto.js is the offline-first client for Kinto.

With the help of Babel and browsersify, a bundle is generated for Firefox with the minimum transpilation possible (eg. CommonJS require, ES7 decorators).

kinto.js

From the kinto.js repo, generate the moz-kinto-offline-client.js file:

$ npm run dist-fx

And overwrite it in the Firefox code base:

$ cp dist/moz-kinto-offline-client.js ../mozilla-central/services/common/kinto-offline-client.js

kinto-client.js

From the kinto-client.js repo, generate the moz-kinto-http-client.js file:

$ npm run dist-fx

And overwrite it in the Firefox code base:

$ cp dist/moz-kinto-http-client.js ../mozilla-central/services/common/kinto-http-client.js

Run the tests

First, follow the instructions to build Firefox.

For JavaScript updates only, have a look at Artifacts Builds, trading bandwidth for compilation time.

$ ./mach build faster
$ ./mach xpcshell-test services/common/tests/unit/test_kinto.js
$ ./mach xpcshell-test services/common/tests/unit/test_storage_adapter.js

Or all tests related to Kinto:

$ ./mach xpcshell-test services/common/tests/unit/*into*

Gfx blocklist tests

The Gfx test suite requires the debug mode to be enabled. Add this to mozconfig file, rebuild and run the tests:

ac_add_options --enable-debug

Debug content signature

You can get tests (or Firefox) to give you more information on what the content signature verifier is doing by setting the NSPR_LOG_MODULES environment variable. For example:

EXPORT NSPR_LOG_MODULES=ContentSignatureVerifier:5,CSTrustDomain:5

TDD mode

Using inotify, we will detect a file change in the dist/ folder and run a series of commands to execute the tests automatically.

First, install inotify-tools:

sudo apt-get install inotify-tools

Then start an infinite loop with inotify-wait:

while true; do
    # Wait for a change
    inotifywait -q -e create,modify,delete -r ~/Code/Mozilla/kinto.js/dist
    # Execute these commands
    cp ~/Code/Mozilla/kinto.js/dist/moz-kinto-offline-client.js services/common/kinto-offline-client.js
    ./mach xpcshell-test services/common/tests/unit/test_storage_adapter.js
    ./mach xpcshell-test services/common/tests/unit/test_kinto.js          
done

Source: Antoine Cezar

Submit patch

> Patch are contributed to kinto.js and kinto-client.js, which are first released on NPM.

DO NOT land files that are not tagged officially on upstream repositories.

Become a contributor

• Generate a SSH key
• Open a bugzilla ticket to request Level-1 access
• https://www.mozilla.org/en-US/about/governance/policies/commit/access-policy/
• Get a vouch (*optional for level1*)

Configure SSH key for hg:

Host hg.mozilla.org
  User user@server.com
  IdentityFile ~/.ssh/contrib_moz

Run integration tests: «Try»

See https://wiki.mozilla.org/ReleaseEngineering/TryServer

Or use a gecko-dev fork from Github, and with install moz git tools

git push-to-try -t --rev master..HEAD ~/hg/mozilla-central/ -b do -p linux,linux64,macosx64,win32,win64 -u xpcshell -t none

Submit for review

See http://mozilla-version-control-tools.readthedocs.org

# Commit with link to Bugzilla
hg commit -m "Bug XXXXX - Upgrade <lib> to X.Y.Z"

# Submit to MozReview
hg push review

# Keep a bookmark of your branch to address review.
hg bookmark bug-XXXXX

# Go back to «master»
hg update central

Examples:

• https://reviewboard.mozilla.org/r/45445/