Confirmed users
971
edits
Firefox Home Server Architecture: Difference between revisions
no edit summary
No edit summary |
|||
| (2 intermediate revisions by the same user not shown) | |||
| Line 21: | Line 21: | ||
This is work in progress. We want to keep the first release simple. | This is work in progress. We want to keep the first release simple. | ||
== | == Web Components == | ||
(Missing: memcache where appropriate) | (Missing: memcache where appropriate) | ||
=== Zeus === | === Zeus === | ||
==== Description ==== | |||
Zeus is the SSL frontend and the load balancer. Ideally Zeus will be able to look at the web service URLs from Firefox Home and based on some value in the request, send the request to the right Web Server. | Zeus is the SSL frontend and the load balancer. Ideally Zeus will be able to look at the web service URLs from Firefox Home and based on some value in the request, send the request to the right Web Server. | ||
| Line 33: | Line 35: | ||
=== Web Server === | === Web Server === | ||
The Web Server runs the Firefox Home web application. The | ==== Description ==== | ||
The Web Server runs the Firefox Home web application. It serves both static content and JSON web services. The site will be very HTML5/JS based, but in a later version we probably also want to serve more static HTML4. | |||
The Web Server also periodically pushes sync requests to the Sync Queue. For example every 15 minutes. | The Web Server also periodically pushes sync requests to the Sync Queue. For example every 15 minutes. It will simply iterate over the sync accounts and based on the status of that account it will push a work request in the sync queue. This can be optimized by syncing less frequently used accounts less often. Or even skip accounts that have been created but not used in a while. | ||
Ideally web servers do not do this kind of work but I think that a simple cron job to queue messages will not cause a major load on these systems. If it does, or when there is a need for more periodic tasks, then we can move those to an extra machine. | Ideally web servers do not do this kind of work but I think that a simple cron job to queue messages will not cause a major load on these systems. If it does, or when there is a need for more periodic tasks, then we can move those to an extra machine. | ||
Each Web Server has a database dedicated to it. The Web Server talks directly to this database. It will need to do real queries. It will of course try to cache as much as it can in for example memcache. | Each Web Server has a database dedicated to it. The Web Server talks directly to this database. It will need to do real queries. It will of course try to cache as much as it can in for example memcache. | ||
==== Software Requirements ==== | |||
The web application will be written in Python. Currently the prototype is a Pyramid/SQLAlchemy type application. Ideally it is hosted on the same kind of platform that Services uses or at least something they approve. | |||
==== Performance Concerns ==== | |||
This application differs from the sync services that it will actually need to run real queries and searches on people's sync data. This will be a very different load/usage pattern. | This application differs from the sync services that it will actually need to run real queries and searches on people's sync data. This will be a very different load/usage pattern. | ||
| Line 70: | Line 80: | ||
=== Sync Queue === | === Sync Queue === | ||
==== Description ==== | |||
The Sync Queue exists to coordinate sync operations. The Web Servers periodically push requests to the queue, which the Sync Workers will pick up. | The Sync Queue exists to coordinate sync operations. The Web Servers periodically push requests to the queue, which the Sync Workers will pick up. | ||
| Line 78: | Line 90: | ||
=== Sync Worker === | === Sync Worker === | ||
==== Description ==== | |||
A Sync Worker is a machine that will sync a user's account. | A Sync Worker is a machine that will sync a user's account. | ||
| Line 87: | Line 101: | ||
The frequency and scheduling of sync operations is not in control of the Sync Workers. They simply handle sync requests. | The frequency and scheduling of sync operations is not in control of the Sync Workers. They simply handle sync requests. | ||
This is an important part of the infrastructure but there is no need to get extremely high uptime on this part. In case of failures we would simply not sync, which is not something the end user would notice in form of data loss. | ==== Software Requirements ==== | ||
The sync client will be written in Python and can be hosted on any flavor of Linux. | |||
==== Hardware Requirements ==== | |||
For these machines, memory is more important than disk. More memory allows us to run more sync processes in parallel. | |||
==== Uptime ==== | |||
This is an important part of the infrastructure but there is no need to get extremely high uptime on this part. In case of failures we would simply not sync, which is not something the end user would notice in form of data loss, but only as a sync that might not have happened on time. | |||
Because the workers are driven by a queue, another machine would simply pick up the sync request the next pass. | |||
(We could also let the workers ack the sync request so that in case they fail, the original request that was in progress goes back in the queue to be picked up by another worker, but i don't think that is even needed here) | |||
=== Public Sync Services === | === Public Sync Services === | ||