Services/F1/Server/Architecture

Goals

• provide a scalable architecture to serve up to 100M users
• make sure the server is immune against third-party failures (Twitter is down)
• isolate parts that need to be specifically secured in our infrastructure

Expected Load

The current load is 50k registered users that perform 1600 requests per day. 0.032 requests per day and user.

If we transpose it to 100M users, the number of requests per day will be 3,200,000 so 37 requests per second.

The architecture we're building should therefore support at least this load. 50 RPS seems a good goal.

Since calls are done on third party servers, there's an incompressible time spent there. One request can last up to 3 seconds. So in order to support 50 RPS, we need to be able to support a real load of 150 simultaneous clients.

data size

The average size of data a user transfers through our servers is 16 bytes per day. So if a queue stores this data, it represents 1.5 Gbyte / day for 100M users.

Synchronous Architecture

The synchronous architecture is composed of two servers:

• server-shared-auth: creates and returns tokens for each service
• server-shared-send: calls the service and returns the response synchronously

The Services Status DB is a key/value storage global to the cluster and replicated in several places. It can be eventually consistent.

Detailed transaction:

1. The server checks for the service status to return a 503/Retry after in case the service is down. . Each service has a (GR / BR) ratio stored in that DB. GR = Number of good responses. BR = Number of bad responses. What are "Good" and "Bad" responses is to the workers discretion.

2. the server calls the service

3. the server upgrades the status of the service, by incrementing the number of good or bad responses.

Load Balancing

A simple way to balance the load is to have servers dedicated to a specific service.

A X-Target-Service header set by the server can define which service is going to be called and a Zeus load balancer can dispatch the request to a specific subset of servers. This will allow a fine-tuning of the pooling capacity of the web server and also will let us specifically add more servers per service.

Asynchronous Architecture

Big Picture

• The Post office Front server (PF) receives a send request.
• The request is pushed into a queue
• A worker (W) picks the job and push the result into a key/value storage
• the client grabs back the result via the Receiver Front server (RF)

Detailed transaction

1. PF checks the status of a service by looking at the Services Status DB. Each service has a (GR / BR) ratio stored in that DB. GR = Number of good responses. BR = Number of bad responses. What are "Good" and "Bad" responses is to the workers discretion.

Given a threshold, PF can decide to return a 503 immediately, together with a Retry-After header.

2. PF creates a unique Session ID for the transaction and stores it into the Session DB.

3. PF push the work to be done, together with the session id, into the right queue. It then return a 202 code to the client.

4. A worker picks the job and do it. Once it gets the result back, it sends back the result in a response queue.

5. A Consumer picks the jobs that are done in the response queue.

6. The consumer selects a Result DB server by using a simple modulo-based shard algorithm. It pushes the result into the select Result DB.

7. The consumer updates the Session DB by writing there the Result Server ID associated to the Session ID. Once this is done, the job is removed from the queue.

8. The consumer updates the Service status DB with the status (G or F) provided by the worker.

9. When the user ask for the result via RF, RF looks in the Session DB what is the Response DB that has the stored result.

10. The RF then picks the result and returns it.

Databases

The Session DB and Services Status DB are key/value stores global to the cluster and replicated in several places. They can be eventually consistent.

The Result DB is a key/value store local to a single server. It holds the result for a specific transaction.

The queues are providing a Publisher/Subscriber pattern, where each worker picks jobs to be executed on a specific service.

Failure Scenarios and persistency

The general idea is that a transaction can be re-tried manually by the user if there's an issue in our system. Although our system should prevent clients to send more work if a third party service is down.

• if PF cannot reach on of those, a 503+Retry after is sent back:
  • the services status DB
  • the session DB
  • the queue

• if the queue crashes, the data stored should not be lost. Workers should be able to resume their subscriptions

• The job should be left in the queue until the transaction is over
  • a worker that picks a job marks it as being processed
  • if a worker crashes while doing the job, the marker should decay after a TTL so another worker can pick up the job
  • if a consumer cannot reach the result DB, the job stays in the queue and eventually goes away after a ttl.
  • if a consumer cannot reach the session DB the job stays in the queue and eventually goes away after a ttl.

• if a consumer cannot reach the services status DB, nevermind.

XXX

Security

XXX explains the level of security of each piece -- who holds the app key, the tokens, etc

OAuth Dance

XXX