View source for Raindrop/FrontEnd

[[Raindrop]]

== General Architecture ==

For a quick summary of Raindrop's complete architecture, see [[Raindrop/SoftwareArchitecture| Raindrop Software Architecture]].

The Front End of Raindrop is the part that constructs the user interface. It calls APIs on the [[Raindrop/BackEnd|back end]] to populate the UI with data. In the [[Raindrop/SoftwareArchitecture| Raindrop Software Architecture]], the files for the Front End live in CouchDB and are served to a web browser.

The user interface is plain HTML, JavaScript and CSS. It is served as static files from CouchDB. Some notes about this:

* Couch needs exact URLs for files, it does not have a concept of mapping a document attachment as the "default" document to serve (in fact that sentence probably does not make sense in CouchDB terms). So, it means the user has to explicitly ask for "index.html" and not just specify a "directory" on the URL.
* Modifying files that also need to be committed to source control means either:
** running the run-raindrop.py script to push the updated files into the couch
** running a reverse proxy via Apache or nginx to load the files from disk, but still route to the couch for API calls.
** in the future, use Bespin to edit the files, and create a Bespin module that knows how to push the code into the couch and into the local code repo?
* Serving the files from CouchDB means they can be replicated to other CouchDB instances. This may be useful in the future.

As much as possible, HTML, JavaScript and CSS are used to their respective strengths:

* Styling info is kept in CSS files, with JavaScript only applying new class names to DOM nodes. If JavaScript does touch a DOM node's style, it is only to hide or show an element. Inline style tags in the HTML are strongly discouraged.
* HTML is used where possible to specify DOM structure instead of using JavaScript to build up complex DOMs. The widgets can contain a string of HTML for their template, or better yet, reference an HTML file that has the template.
* Inline JavaScript is avoided in the HTML where possible. Exceptions might include preventing form submits via an inline onsubmit="return false", but that should be it.

The Dojo JavaScript Toolkit is used to construct the Front End's core. Dojo is used because it has:

* a module loader that works in JavaScript. This makes it easy to dynamically load extensions and UI components on demand.
* a module/component model with inheritance. This has already been used to provide quick protopying of alternate UIs as well as making extensions easier by giving proper function endpoints for extension attachment.
* real HTML for widget templates, which is easier to follow and modify by non-JavaScript experts.
* i18n (internationalization) support built-in, so it is easy to make UI widgets that work for any language.
* a build system that allows bundling the modules for an application into just a couple of optimized files, to improve the end user experience.
* a rich set of open source, CLA-safe modules.

While the UI widgets and the base infrastructure uses Dojo for the reasons above, jQuery is included in each page, mainly for benefit of extension developers.

=== Code Layers ===

The Front End has the following logical sections:
# '''JavaScript Data API''' files that wrap the sometimes-complex chain of CouchDB calls (in raindrop/client/lib/rd/api)
# '''UI Widgets''' that use the JavaScript API (in raindrop/client/lib/rdw)
# '''Web Applications''' that use the UI Widgets (in raindrop/client, the directories that are not "lib", "dojo" or "bespin".

=== JavaScript Data API ===

Right now, the JS API layer is split between two different styles: the older, explicit callback style that is in raindrop/client/lib/rd/ (the contact.js, conversation.js store.js and tag.js files) and a newer API style that uses Deferreds in raindrop/client/lib/rd/api/.

The goal was to move to the second, Deferred-based style, but the work is not complete.

The Deferred-based style uses files in raindrop/client/lib/rd/api/. All those files add methods to an rd.api.Api() object. The goal is to allow chained API calls through the use of dojo.Deferred objects. It is not clear if this is the right model for the calls. It may be the right model, but may just mean the normal use case is very small chains, like this:

<pre>
rd.api().conversation({
  ids: []
})
.ok(function(json) {

})
.error(function(err) {

});
</pre>


Ideally the API will match well with the [[Raindrop/WebApis|Web APIs]] we eventually want.

'''How to get involved with JS Data APIs: ''' Post a message or search the archives on the [http://groups.google.com/group/raindrop-discuss mailing list] with "JS Data API:" in the Subject.

=== UI Widgets ===

The UI widgets use the JS Data API to display some aspect of the back-end data. They are located in raindrop/client/lib/rdw/ directory.

Most widgets derive from the rdw._Base base "class". Dojo's Dijit widget system is used for the basis of the widgets. rdw._Base uses dijit._Widget and dijit._Templated. It also loads a default i18n bundle and creates some utility methods for dealing with child widgets.

It is very important to support an extension editing experience that allows an extension developer an easy way to make changes in their extension and see immediate results. This works in Raindrop by by destroying and recreating widgets on the fly, using data from the old widgets. It is very important to clean up any DOM nodes or state information that should not apply to the recreated widget. So most widgets extend the destroy() method to do this cleanup.

==== Directory Structure ====
Some notes about the directories under "rdw":

* css: holds the CSS for individual widgets. It should be possible for other developers to create new CSS for the widgets, but a default set is available in this directory. These files might be moved to some other area at some point.
* ext: this holds a set of default extensions. They may be moved out to some other place at some point.
* nls: contains the i18n bundles. Right now there is only one, and only for the "root" locale. Dojo's i18n system is used to load this bundle.
* resources: auxiliary files that are not JavaScript, CSS, HTML. They should not be specific to any particular CSS theme.
* story: a set of rdw.Story subclasses that handle display of specialized story types.
* template: the HTML templates for some of the widgets. The widgets use templateString with dojo.cache to load these templates. The templates can be inlined with a custom Dojo build process.

==== Logical Structure ====

rdw.Stories is a central widget that handles getting a list of conversations then displaying the conversations using an array of rdw.Story objects. rdw.Stories has two different modes: the main "inflow" view and the specific list view. Both the inflow and specific list views can trigger the rdw.Stories object to view a full conversation instead of a list of summarized conversations.

The inflow view of rdw.Stories uses some grouping widgets to group some messages together into even more compact widgets at the bottom of rdw.Stories widget. rdw.Stories favors direct and person-to-person group conversations over these compact group displays. New compact group widgets can be registered with rdw.Stories via the homeGroups property on rdw.Stories. rdw.Story.TwitterTimeLine, rdw.ext.MailinList, rdw.ext.facebookNotification, rdw.ext.feedNotification, and rdw.ext.twitterNotification all register with rdw.Stories' homeGroups to get a shot at grouping messages.

The items in rdw.story and rdw.Message are mainly there to support rdw.Stories and its use.

The other widgets are mostly self-contained and should be fairly self-descriptive. If you look at one of the applications, like raindrop/client/inflow, you can see how the widgets are grouped together.

==== Design Points ====

Extensibility is very important for the widgets, so as mentioned above, destroying DOM Nodes or state that should not be transferred to recreated widgets is very important.

Similarly, the widgets should not hard code a reference to another widget where possible, but instead use a string to represent the name of the widget to use. Then, that widget should be lazy loaded via dojo["require"]/dojo.addOnLoad calls inside the widget. This allows for mixing in replacement widgets without too much trouble.

The widgets make a point of not setting any CSS style properties directly, except maybe display and visibility, for hiding and showing things. Some things are also set by animations, but the animation effects should be removed at the end of the animation. The widgets instead favor adding and removing class names that represent changes in the widget state.

Publish and Subscribe (pub/sub) is used extensively in the system, via rd.pub() and rd.sub() (uses dojo.publish and dojo.subscribe underneath). Many widgets, like rdw.Stories and rdw.Summary, use an array of topic names to register for topics, and extensions can add to these lists of topic names. pub/sub should be favored when loose coupling is desired for notification of state changes or actions.

Widgets should favor using <a> tags with href's that have fragment IDs (#hashurls) to communicate actions. If the widget supports many actions in this way, event delegation should be used to catch the click events and dispatch to internal functions via the anchor tag's fragment ID href.

There is a document-level handler built-in than can convert #rd-name hrefs into rd.protocol.name published topics. This allows the URL of the page to reflect the state and still allow the use of publish/subscribe to wire up listeners for that state change. This makes it easy to trigger new state changes without needing to explicitly define a click handler in a widget.

The widgets are designed specifically for the needs of the inflow application (described in next section). The goal is to not make extremely generic widgets as the default set for raindrop, but to design them around problems we want to solve via an inflow application. It is possible to create your own widgets, as part of an extension or as part of a new application, but the ones shipped in the raindrop/client/lib/rdw directory will be tailored to meet inflow's needs.

'''How to get involved with UI Widgets: ''' Post a message or search the archives on the [http://groups.google.com/group/raindrop-discuss mailing list] with "UI Widgets:" in the Subject.

=== Web Applications ===

The highest level of the code structure is applications. They are HTML, JavaScript and CSS files that reside in the rdw/client folder. Right now, the following apps have been started:

* '''raindrop/rdw/client/extender/''' - The UI for a user to create simple extensions and try them while they edit.
raindrop/rdw/client/inflow/''' - The main application for raindrop -- shows your messages with a focus on conversations and person-to-person interaction.
* '''raindrop/rdw/client/inflowgrid/''' - An experiment that explores an alternate presentation of the inflow app using a strict grid layout.
* '''raindrop/rdw/client/rd_list/''' - A listing of installed applications. Not very interesting at the moment, but may gain more importance as other apps are written for the raindrop platform.
* '''raindrop/rdw/client/settings/''' -  Shows the user's accounts, allows them to enter credentials for each account.

All the apps follow a basic structure:

An '''index.html''' that defines the start page of the application. It may lay out some UI widgets in the markup. It will likely be very simple, with just a reference to its css file, loading of /raindrop/lib/rdconfig.js (mandatory for the app to work with extensions), and a script tag doing an rd.require() call for the app's top-level JavaScript file.

A '''CSS file''' that may have @import calls to pull in CSS for specific widgets. These @import calls may be optionally inlined for app distribution by a (to be built) app bundling system.

A '''JavaScript file''' that does the dojo.require() calls for widgets and JS Data API modules, and contains any bootstrap logic for the app. The dojo.require calls may be optionally inlined for app distribution by a (to be built) app bundling system.

'''How to get involved with Web Applications: ''' Post a message or search the archives on the [http://groups.google.com/group/raindrop-discuss mailing list] with "Web Apps:" in the Subject.

== Extensions ==

Extensions are an important part of the Raindrop platform. They allow customizing the behavior of Raindrop to fit the user's needs, and it gives other users and developers an easy way to participate in the platform.

Supporting Extensions has an implication on the construction of UI Widgets (see that section above), since we want to allow in-place updates of extension changes as the developer makes changes.

There are some extensions in the raindrop/client/lib/rdw/ext directory. These may be moved out at some point to a raindrop/client/ext directory. You can look at those extensions to get a feel for how they are constructed. the MailingList one is interesting because it extends a few widgets.

It is likely that most extensions will want to extend an existing module, so the extension model is focused on attaching methods and properties to existing modules. However, it is possible to create an extension that just load a new, independent component. In that case, rd.applyExtension() will not be called.

This may change over time, but as of right now, the basic construction of an extension is shown below, called "sample". An extension can register certain types of extension points, all listed below: "before", "after", "around", "replace", "add" and "addToPrototype". Typically an extension will only use one or two types of extension points.

<pre>
//Tell the loader that rdw/Message will be modified,
//and give the name of this extension. Also indicate
//the dependencies for the extension.

run.modify("rdw/Message", "ext/sample",
["rd", "rdw/Message"], function (rd, Message) {

    //Tell raindrop you want to apply an extension, and what will be extended.
    rd.applyExtension("ext/sample", "rdw/Message", {

        //Register "before" extension points.
        //These extension functions should be called before
        //the matching function on rdw/Message
        before: {
            //This postCreate will be called before an instance of
            //rdw/Message's postCreate method is called.
            postCreate: function() {
            }
        },

        //Register "after" extension points.
        //These extension functions should be called before
        //the matching function on rdw/Message
        after: {
            //This postCreate will be called after an instance of
            //rdw/Message's postCreate method is called.
            postCreate: function() {
            }
        },
        
        //Register "around" extension points.
        //These extension functions should be called before
        //the matching function on rdw/Message, and this extension 
        //point has the option to to call the original rdw/Message 
        //function or bypass it completely. It also has the ability
        //to modify the return value from the rdw/Message function call.
        //This type of extension point is very experimental and may 
        //change due to its use of arguments.callee
        around: {
            //This postCreate will be called before an instance of
            //rdw/Message's postCreate method is called. If it wants to
            //call rdw/Message's postCreate function, it can 
            postCreate: function() {
                //Demonstrates calling the rdw/Message function 
                //that this extension point is wrapping around.
                var value = arguments.callee.target.apply(this, arguments);
                //Modify value then return it.
            }
        
        },
    
        //Register "replace" extension points.
        //These extension functions replace the the matching function
        //on rdw/Message. It will first check rdw/Message for 
        //the matching function, then look at rdw/Message's prototype
        //for the method.
        replace: {
            //This postCreate will be replace rdw/Message's
            //prototype.postCreate
            postCreate: function() {
            }
        },
    
        //Register "add" extension points.
        //These extension functions will be added to rdw/Message.
        //Note that they are added to rdw/Message, and *not*
        //rdw/Message's prototype, so they will not be callable
        //from instances of rdw/Message.
        add: {
            //This function will be placed on rdw/Message,
            //it is like adding a "class-level"
            //function vs an instance-level function.
            getInstances: function() {
            }
        },
    
        //Register "addToPrototype" extension points.
        //These extension functions will be
        //added to rdw/Message's prototype, and *not* to rdw/Message.
        addToPrototype: {
            //This function will be placed on 
            //rdw/Message's prototype, it is like adding an
            //instance-level function vs a "class-level" function.
            onActionClick: function() {
            }
        }
    });
});

</pre>


rd.applyExtension() take the name of the extension so it can dynamically enable or disable it on the fly. For example, if the user decides to disable it via the Extender web app. rd.applyExtension() actually creates another function wrapper that checks the extension registry for each call, to make sure the extension is enabled. If it is, then the extension function is called. If not, then it is just skipped.

Extensions are only loaded when the module they are extending has been loaded.

=== Still to do ===

Right now, if the extender web app is used, it only saves extensions in CouchDB, so if the CouchDB database is deleted, then the extension is lost. We need a nice way to export an extension to disk so it can be put into source control. Perhaps this will fall out naturally once we work out extension distribution.

Extensions can add CSS to the page by using rd.addStyle(). It uses the logical dot-name, for example, rd.addStyle("ext.foo.main") will load ext/foo/main.css in a link tag. Ideally this CSS could be inlined via a distribution tool, but there are path implications for url() references in the CSS.

Currently the metadata on an extension is pretty light: no author or version information is stored. We also need to allow for registering extensions so that they only apply to specific web apps vs. an extension that applies for all web apps.

Distribution of extensions still needs to be defined. Something safe and secure would be nice.

'''How to get involved with Extensions: ''' Post a message or search the archives on the [http://groups.google.com/group/raindrop-discuss mailing list] with "Extensions:" in the Subject.