Firefox Home Server Architecture

Players

(Missing: memcache where appropriate)

Zeus

Zeus is the SSL frontend and the load balancer. Ideally Zeus will be able to look at the web service URLs from Firefox Home and based on some value in the request, send the request to the right Web Server.

(Home can of course also ask services to what server to go, just like is done for sync clients now.)

Web Server

The Web Server runs the Firefox Home web application. The web application will be written in Python. Currently it is a Pyramid/SQLAlchemy type application. Ideally it is hosted on the same kind of platform that Services uses.

The Web Server also periodically pushes sync requests to the Sync Queue. For example every 15 minutes.

Ideally web servers do not do this kind of work but I think that a simple cron job to queue messages will not cause a major load on these systems. If it does, or when there is a need for more periodic tasks, then we can move those to an extra machine.

Each Web Server has a database dedicated to it. The Web Server talks directly to this database. It will need to do real queries. It will of course try to cache as much as it can in for example memcache.

This application differs from the sync services that it will actually need to run real queries and searches on people's sync data. This will be a very different load/usage pattern.

Database

The databases contain the following:

• Structured sync data (bookmarks, tabs, history)
• Account information (credentials, bulk keys)

One Web Server / Database combo handles a certain amount of users.

Data is stored unencrypted in the database.

There is currently no preference for any specific type of database. MySQL, Drizzle (is the Shizzle) or anything that allows queries and joins will be fine.

Data on this server is a *copy* of what already exists in the Sync Service. I would therefore say that there is a much lower requirement to keep this running. In case of problems we can always sync the data again. This will be a relatively expensive operation but it should be pretty fast.

(Except for the user's credentils and key. Maybe those need to move to a secondary system?)

Sync Queue

The Sync Queue exists to coordinate sync operations. The Web Servers periodically push requests to the queue, which the Sync Workers will pick up.

I understand that we do not currently use any specific queue system. Anything is really fine, as long as it is relatively simple to talk to it from Python.

I do not think queue persistence is needed for this application. If the queue gets lost then we simply schedule a request to sync within the next 15 minutes or whatever the sync interval is.

Sync Worker

A Sync Worker is a machine that will sync a user's account.

To make this part of the infrastructure more reliable and distributed, the workers listen to a work queue. Each worker can sync N accounts in parallel.

The worker takes a job from the queue and then talks to the database where that user's data lives and to the Public Sync Services and sync data.

The frequency and scheduling of sync operations is not in control of the Sync Workers. They simply handle sync requests.

This is an important part of the infrastructure but there is no need to get extremely high uptime on this part. In case of failures we would simply not sync, which is not something the end user would notice in form of data loss. The data would simply not be up to date.

Public Sync Services

Simply the sync services as they exist now. Ideally no changes are needed and Firefox Home is simply another client that accesses sync using the defines APIs that all other clients use.

There is no special need for shortcuts to the Sync Services on an infrastructure level. As long as the Firefox Home Sync Workers can talk HTTPS to the Sync Services, all is good.

Pretty pictures