Raindrop/WebApis
This outlines our current thinking and dilemmas regarding Raindrop Web APIs. We welcome all contributions and input into what we have described here.
High level goals
There are lots of reasons to have web based APIs to a messaging system like Raindrop. Here are some of them. Note that some are platform requirements, some are more product/application specific. We may need multiple APIs to support the different goals:
Modularity
Having well defined APIs would allow modularity between the various tiers of the system, from the message aggregation and back-end extension engine (currently CouchDB-based), to the conversation-aggregation layer (in the client code as of this writing), to the front-end UI code (in the client). Having APIs would allow us to be resilient to changes in CouchDB, to considering alternatives in any of the tiers, etc. It's just good software engineering.
Lighter weight clients
The current APIs require the client to do a lot of work, both in terms of processing power, and in terms of HTTP connections to the database. By moving some of that work behind an HTTP end-point, we should be better able to support clients on less capable platforms (e.g. phones), the performance of the client should improve, and in some installations we should be able to deploy web server technologies like caching layers to improve performance further.
Hackability
We have two APIs, right now, the direct CouchDB calls, which are complex due to the inherent complexity of the megaview (due to its genericity), and the very app-specific JavaScript APIs in the inflow code. Combined, it becomes harder to try new things like a non-JavaScript front-end.
Support and develop open standards
We want to help define open standards around messaging. We want to use current best practices in our domain - of which the only real contender is a REST based API accessed over HTTP. The API must allow for high-bandwidth low-latency applications.
Simplicity
The API should deal in the same concepts as the application rather than the details of the implementation; for example, messaging applications want to work with message related objects, not with database documents. The API should be simple enough to allow designers to easily experiment with concepts without needing them to understand all the technical details of the platform.
On simplicity...
While many of the above goals are self-explanatory, there are a number of aspects relating to the simplicity of the API that deserve further discussion.
Up in the sky, is it an application, is it a platform? No, it's a raindrop!
What exactly is a 'raindrop application'? The inflow application we ship with raindrop is intended to be just one of many possible raindrop applications. We expect people will innovate with their own applications, and while these are likely to be vaguely 'message based', they may deal with concepts and metaphors which are foreign to our inflow application.
This implies that we may need a sense of layering in the API itself, so each application can provide an application specific API built around some lower-level constructs provided by the platform. In other words, this API itself needs to be hugely extensible and probably based around a similar 'extension' model that raindrop itself uses.
Page based applications
Most raindrop applications are likely to be based in a browser, and is likely to have the concept of 'pages' (even if there is only 1 page). For applications which offer a number of pages (eg, the inflow application will have as many pages as necessary to show all your messages), it is very rare that a user will venture past the first couple of pages. If a user can't find what they want in those first few pages, they are likely to attempt a different 'vector' to locate what they want (such as searching, filtering, etc.)
A complication here is that exactly what is on a page depends on a number of factors - eg, the current filter being applied, even the size of the browser window. A simple user-interface action may drastically alter the items which appear on a given page.
A consequence of this is that:
- The API should, as much as possible, be based around the concept of pages.
- The API should be optimized around the idea that only the first couple of pages are likely to be hit. The first couple of pages must be able to be fetched and rendered quickly, whereas the user and application will be more tolerant of delays in rendering later pages.
- We need to think more about how the API can determine what a "page" really is, or a better model for implementing this requirement.
Extensible Applications
Raindrop has been designed to be highly extensible. Third parties can easily and simply add extensions to raindrop which add further attributes to contacts or messages. This introduces come conflict with the requirement to deal with 'application concepts' - we must design our API such that it works with entities like 'message' or 'conversation', while still allowing for extensions to change exactly what attributes and properties these entities possess.
The 'inflow api'
Once consequence of the above discussions is that each application is likely to require its own application specific API. This section is a place-holder for the API provided by the 'inflow' application.
This describes the (future) API for the 'inflow' application. The inflow is the test-bed against which our ideas will be pitted. This needs to grow into a concrete API proposal which can be used by the inflow.