Education/Projects/ProcessingForTheWeb

This is a working draft describing a proposed project to bring a complete implementation of the Processing language to the open web.

Summary

Processing for the Web is a proposed Mozilla-sponsored initiative to create a complete and robust port of the Java-based Processing language and environment (popular for art, design, and visualization applications) to the open web, using standard technologies such as JavaScript, Canvas, etc.

The goals motivating the initiative are as follows:

• to enable Processing users to leverage the full power of the open web, and help spark a new wave of innovative uses of Processing
• to serve as a forcing function to drive improvements in key aspects of Mozilla products and technologies, including JavaScript performance, Canvas functionality, applications of Bespin, and others
• to reach out to and increase cooperation with the design, art, and new media communities, both as part of the existing Mozilla Education program and as a key component of a proposed Mozilla Media program

The initiative will incorporate the following activities:

• extending and improving the core functionality of the current processing.js port of Processing to JavaScript
• implementing a web equivalent of the current Processing development and run-time environment, with additional features to leverage the full power of the web (e.g., to support collaborative development)
• encouraging creation of new Processing applications explicitly designed for the web
• improving the underlying Mozilla infrastructure technologies required to support a powerful, robust, and conformant Processing implementation for the web

Some activities in the initiative, in particular working on the required Mozilla technology infrastructure, would be carried out by existing Mozilla core contributors. Other activities would be done in cooperation with others; in particular we propose to package many of the needed tasks as student projects to be done by design, programming, and other students under the auspices of the Mozilla Education program

What is Processing?

To quote from the Processing project home page:

Processing is an open source programming language and environment for people who want to program images, animation, and interactions. It is used by students, artists, designers, researchers, and hobbyists for learning, prototyping, and production. It is created to teach fundamentals of computer programming within a visual context and to serve as a software sketchbook and professional production tool.

Processing arose out of work in the Aesthetics and Computation Group at the MIT Media Lab. It has attracted a user base of artists, designers, people doing data visualizations, and others.

Processing language syntax is based on Java, and the original Processing environment requires an underlying Java run-time. The Processing language includes standard functions for doing drawing and related operations, and the Processing environment offers a straightforward and simple way to edit and run Processing programs ("sketches") without the complexity of Java compilers and other tools.

Why Processing for the Web?

Why re-implement Processing in the context of the web? Given Processing's Java syntax and heritage, wouldn't it make more sense to encourage people to move to using web technologies like JavaScript and Canvas directly, perhaps supplemented by JavaScript toolkits where appropriate? And even if porting Processing to the web is a good idea, why should Mozilla get involved?

For the reasons discussed below, we believe that this project can bring real benefits to the Processing community, to Mozilla products and the Mozilla community, and ultimately to the open web.

It's good for the Processing community

Processing has a large and enthusiastic community of people who have demonstrated a great deal of creativity and innovation. Doing a complete and robust implementation of Processing using web technologies benefits that community in multiple ways:

• It helps get more people involved in using Processing by lowering the barrier to entry: potential Processing developers would not have to download and install the Processing envronment, but could develop and run Processing sketches directly in their browsers. The resulting sketches could also be viewed in any modern browser, without the need for browser Java plugins.
• It would help enable new types of Processing uses that leverage the power of the web, for example to access web-based data and services, enable web-based user interaction and collaboration, and so on.
• Last but not least, it could also help enable easier collaboration on development of Processing applications, given suitable support in an underlying web-based Processing development environment.

A high-quality web-based Processing implementation could help spark a new wave of innovative uses of Processing. These would be valuable both for their own sake and also for demonstrating and promoting the power of the open web technologies that made Processing for the web possible.

It's good for Firefox and other open web products and technologies

The Processing for the Web initiative could serve as a forcing function to drive improvements in key aspects of Firefox and other open web products and technologies from Mozilla and others.

• JavaScript performance. In order to compete with the traditional Java-based Processing implementation the web-based implementation of Processing will need to have performance for basic language operations that is comparable to that of Java—which means comparable to C/C++ performance, given the capabilities of today's Java compilers. Recent advances in JavaScript performance provided by the Tracemonkey, Squirrelfish, and V8 JavaScript engines have helped narrow the performance gap in modern browsers like Firefox, Safari, and Chrome. The process of testing a wide variety of existing Processing applications on a JavaScript-based Processing implementation will likely provide a wealth of information on potential areas where further performance improvements are needed.
• Canvas functionality and performance. The heart of Processing is image creation and manipulation; the current Processing implementation features no less than four different rendering modes to enable developers to achieve the fastest graphics performance possible for their particular uses of Processing. Adopting Canvas as the underlying graphics interface for Processing on the Web could help drive improvements to Canvas performance and functionality (e.g., adding 3D support).
• Applications of Bespin and similar technologies. As noted above, one of the nice things about the Processing environment is that it offers a simple straightforward interface for novices to start creating Processing "sketches". Any Processing for the Web implementation will need to duplicate this ease of use, and could also add other features enabled by the web environment, such as storing code in the cloud and providing easy but powerful ways to collaborate with others on sketch development. Processing on the Web could be a significant "customer" for Bespin, "a Mozilla Labs experiment that proposes an open, extensible web-based framework for code editing", and could help spur Bespin's transition from an experimental prototype to an actual product.

Just as a previous generation of advanced web applications drove browser vendors to improve browser performance and functionality, we believe that Processing for the Web can be a real-world application that can spur Mozilla and other browser vendors to provide solid high-performance implementations of the HTML5 family of technologies.

It's good for the Mozilla community

As has been said many times, Mozilla products like Firefox are not ends in themselves but rather a means to an end, namely creating an open web that is open, transparent, participatory, distributed, and hackable, and otherwise embodies the ideals of the Mozilla Manifesto.

In order to achieve this goal we need to continue to grow and strengthen the Mozilla community itself, in order to increase our ability to create new products and technologies for the open web, as well as reach out and work with other communities with similar goals, in order to grow the constituency of people leveraging the open web and promoting its values.

The Processing for the Web project could help achieve both these goals:

• Computer science and programming students could work on a wide variety of student projects related to Mozilla platform infrastructure needed for Processing for the Web. These would include not just coding but also performance testing and bug fixing, technical documentation, etc.
• Due to the roots of Processing in the arts and visualization, the Processing for the Web project offers an entry point into the Mozilla project for students studying design and related topics. This includes not only creating and testing new Processing applications on top of Processing for the Web, but also helping to design and build the user experience for the Processing on the Web development environment.

Processing for the Web offers an opportunity to strengthen existing Mozilla Education initiatives directed at faculty and students in computer science programs, and also to leverage Mozilla Labs initiatives (most notably the Design Challenge series) of interest to faculty and students in design programs. The Processing for the Web program could also be a key initial component in a possible new "Mozilla Media" program to reach out and work with artists, filmmakers, musicians, and other creative individuals working with the open web and open media.

Technical strategy

Relevant technologies

The starting point for Processing on the Web is the existing processing.js JavaScript port of Processing originally created by John Resig of Mozilla and now being developed by a growing community. Processing.js provides support for the Java-based Processing language syntax, along with an implementation of the various Processing library functions.

Processing.js is usable for many Processing applications but is not yet fully compatible with the original Processing implementation. It also does not provide a web-based development environment, though projects like Robert O'Rourke's HasCanvas provide an initial take on what such an environment might look like.

The Bespin project could provide the underlying technology to support online creation and editing of Processing code. The challenge is to provide a Processing for the Web environment that is simple enough for complete novices to pick up quickly, compatible enough with the original Processing environment that existing Processing users can be immediately productive, and powerful and extendable enough to provide good support for advanced tasks such as debugging and performance optimization.

Other relevant projects

OpenProcessing allows people to share their Processing sketches ("a Flickr for Processing", to paraphrase one comment about it). It is an interesting example of a site that likely could benefit greatly from a Processing for the Web implementation.

OpenCode is an attempt to create a web equivalent of the Processing desktop environment by using a server-side implementation of the Processing language combined with a web front end. To quote from the website, "What Processing has done for simplified graphics programming, OpenCode hopes to do for simplified access and interaction with Web-accessible content." It can be thought of as a Java-based Processing environment "in the cloud", including the ability to extend the Java back end to add custom libraries.

Notes

This section is for reviewer feedback, suggested revisions and additions, etc.

• This project could be renamed "Processing for the Open Web" (or "Processing On the Web"), with the acronym "POW" (or even "POW!" :-)
• It's been proposed that as much of the work as possible be done by students under the auspices of the Mozilla Education program. We need a good analysis of the various work that would be required, and how much could be "outsourced" to students.