CloudServices/FirefoxMobileServices/ChannelService

Last updated: 2014/06/06

	THIS PAGE IS A WORKING DRAFT
	The page may be difficult to navigate, and some information on its subject might be incomplete and/or evolving rapidly. If you have any questions or ideas, please add them as a new topic on the discussion page.

Overview

Channel Service is a service for Mozilla Cloud Services that need to communicate with Firefox on the client-side (FxOS, Desktop, etc). It unifies bidirectional communication between the client and Mozilla Cloud Services.

Project Contacts

Principal Point of Contact (US) - Ben Bangert bbangert@mozilla.com

Principal Point of Contact (EU) - Tarek Ziade tziade@mozilla.com

Goals

To develop a single multiplexed communication channel between clients and Mozilla Cloud Services.

This will result in:

• Client-side API for client-side code that needs to talk to a Mozilla Cloud Service
• Server-side API for Cloud Services that need to talk to a client
• Cleaner code in the client that is not entangled with a specific service
• Cleaner client-side service code that does not need to concern itself with the channel back to Mozilla Cloud Services
• Easier updates of client-side code that is not intermixed with channel code
• Ability to update channel code for efficiency/performance/cost-savings without changing other client-side or Cloud Services server-side code
• Isolates the scalability challenge of holding open vast amounts of connections to ChannelService, simplifying development of additional services

Use Cases

Push

The first candidate for using the ChannelService is Push, which is already on FxOS. At the moment the code handling the socket connection is mixed in with the code handling the DOM API's for Push. By splitting the channel out, future Push updates could be easier to land in the client. The server architecture will also be drastically simplified by no longer needing to handle the scaling challenge of all the socket connections.

Presence

The Presence project would benefit from having an easier client-side API to use for talking over a single shared channel.

Loop

Loop's server-side architecture is more complex than necessary because the only way to currently wake a FxOS client is SimplePush and it is unable to carry data (the token). ChannelService would make it easier/faster to deploy client-side code that can get the token simplifying the architecture on the server-side and the requirements substantially.

On the desktop, SimplePush is not available yet, and there is no channel that can be used.

Requirements

Firefox OS

• Client-side JSM developer familiar with Push's implementation that can split out the socket handling portion
• Client-side Push JSM developer to refactor the client-side portion to use the new FxOS Channel Service

Firefox Desktop

• Platform dev's that can land the client-side channel handling code

Fennec / etc

• Same as for Firefox Desktop

Server-side

• Server-side dev's to implement the new Mozilla server-side portion
• Server-side Push dev's to restructure Push to utilize the Mozilla Cloud Channel Service

Design

API

Client

Clients (Firefox, FxOS, Fennec, etc.) will get a new internal API to register for usage of the Channel, events, and to send/receive messages over the channel.

The Client-side Channel API will allow other Mozilla service code running in the client to hook in via Events and direct calls:

Events (to register for):

• OnConnect (called anytime the connection is established)
• OnDisconnect (called if the connection is interrupted)
• OnMessage (called when a message is received over the channel for this service)

API Calls:

• Send(payload, serviceName) - Send the payload over the channel. It will end up in the server-side service handling it. (ie. if push.jsm wanted to send a call to the Push service, it would call Send('whatever', 'push') )

The channel code will initially connect and call 'register' to get a deviceID/key assigned to it. It will need to save both the deviceID/key. On reconnects the client will call 'authorize' with the deviceID/key before it will be considered a valid client. The key acts as a password and is not shared.

If no client-side code registers for Events, the client will not open a channel to Mozilla as no client-side services need to utilize the channel.

Overview of client architecture (note that the API referred to here applies to the XPCOM/IAC chunks):

Server-side Services

Cloud Services can utilize an API to communicate with clients, based on message-passing from the outbound router and inbound relay.

The inbound message queue for a Cloud Service contains the body of the message and metadata:

Type: Data
Metadata:
  DeviceID: AABCDC-550e8400-e29b-41d4-a716-446655440000
Payload: ................

The payload is an opaque blob that should be significant to the Cloud Service utilizing the channel.

The outbound router for a Cloud Service contains the body of the message and metadata on what device the message is intended for. If its not possible to deliver the message immediately (client disconnect, etc.) then the message will be rejected.

Outbound message:

Metadata:
  MessageID: ce3488ea-cf7c-41c3-8110-9907b1fe80e8
  DeviceID: AABCDC-550e8400-e29b-41d4-a716-446655440000
Payload: ................

Outbound router response messages:

Unavailable: ce3488ea-cf7c-41c3-8110-9907b1fe80e8

This indicates that the message of the given MessageID could not be delivered at this time. In the event the DeviceID is not valid than it will be 'UnknownDeviceID' with the device ID as the message. Every message will generate a response indicating if it was successfully delivered or not. More messages may be transmitted at once and will be processed at once, in such a case responses may occur out of order.

In addition to basic bidirectional messaging, ChannelService's inbound relay may contain DeviceID change messages.

If a device changes its DeviceID the Cloud Service should account for this change.

Device Change Message:

Type: DeviceIDChange
Metadata:
   OldDeviceID: AABCDC-550e8400-e29b-41d4-a716-446655440000
   NewDeviceID: ACDCBA-550e8400-e29b-41d4-a716-446655440000

It is not expected that device ID's will change frequently, as a change indicates a client has changed geographic regions (North America -> South America, Europe -> North America, etc). Device ID's may change more frequently in the future if the geographic region is further confined (East Europe <-> West Europe).

Platform Requirements

Firefox OS Modifications

Currently there's a PushService in the parent process. This proposal changes the structure.

• A thread will be launched from the parent process (referred to as the 'Services' Thread)
• The Services thread will have two portions running in it to begin with
  • channel.jsm - Channel handling code, split out from the current PushService.jsm, that handles the connection and implements a basic API for other code that is in the Services Thread
  • push.jsm - Push daemon refactored to utilize the Channel API. PBackground will likely need to be used to have the DOM API call into this thread hanging off the main thread

Cloud Services Channel Service

The Cloud Services Channel Service (fondly pronounced koos-koos) handles the server-side termination of the channel from each device.

Channel Service is deployed as a series of clusters, each managing about 2-3 million clients, with a dozen or so nodes when under full load. Each Cluster results in a DeviceID cluster-name prefix to allow a Service to know which cluster's Outbound Router should handle communications. Services utilizing ChannelService may be deployed as a central service that talks to all the clusters, or could be deployed per cluster if that arrangement is better suited to the service.

The Central Service model of interaction with the ChannelService:

The Distributed Service Model of interaction:

Connection Nodes

These nodes are responsible for:

• Routing inbound-client messages to the appropriate Services Inbound Relay (Push/etc)
• Routing outbound-client messages received from Outbound Routers
• Broadcasting to the Outbound Router indicating what DeviceID's are connected to this node
• Holding very high quantities of connections to clients

By simplifying the responsibilities of the connection node we can easily swap in and replace connection nodes with more efficient implementations as they're available.

Inbound Relay

Inbound relays are responsible for:

• Accepting messages from connection nodes for services
• Spooling a small amount of messages if a service is not available to relay them to

Outbound Router

Outbound routers are responsible for:

• Using their local mapping to determine what ConnectionNode to deliver to for a DeviceID
• Holding connections to ConnectionNode's for message delivery
• Sending messages to deliver to ConnectionNode's and indicating whether they were delivered or not
• Informing the service if a device id is not connected

Push Server Modifications

The Push server service will need to be modified to work with the Cloud Services Channel Service.

Code Repository

Links to the published code bases

Release Schedule

Predicted code delivery dates

QA

Points of Contact

Engineer - Name contact@info

Security and Privacy

Fill out the security & privacy bug template: https://bugzilla.mozilla.org/form.moz-project-review (https://wiki.mozilla.org/Websites/Kick-Off_Form)

For security reviews, there's: https://wiki.mozilla.org/Security/ReviewProcess

Review Status

Bugzilla Tracking # - see https://wiki.mozilla.org/Security/Reviews

Operations

Deployment Architecture

Bugzilla Tracking # -