View source for Gecko:2DGraphicsThoughts

= Gecko Needs =

=== Semantics ===

The basics of cairo's semantics seem to be OK --- fill path/stroke path/showglyphs, with source patterns, destination surfaces, operators, etc.

Some semantic improvements could be made:
* Replace REPEAT/PAD/NONE extend modes with source clipping in tile space (NONE not needed)
* Mask-based complex clipping is really slow and therefore pointless
* Allowing changes to the CTM during path emission is useless for us
* [[Gecko:2DGraphicsThoughts/ErrorHandling]]

=== API ===

Gecko doesn't make much use of the statefulness of contexts. Every time we draw a display item, we reset most of the state. The exceptions are clipping and transforms. (But we often do need to be able to reset clipping to a given clipping state.)

Canvas is different, of course.

Some state is not reset but assumed to be constant between operations: OPERATOR_OVER, default antialiasing mode, etc.

We can do without pushgroup and make do with temporary surfaces quite easily. I don't think we need device offsets.

The context/surface distinction can be simplified. We do need some abstraction for source images that encompasses in-memory buffers, native objects, and temporary surfaces ... maybe these can be different kinds of patterns. But we don't draw into a surface with multiple contexts at the same time. We can get by with context creation functions that create a context for specific targets --- an in-memory buffer, a native target, or a PDF stream --- just like cairo surface creation does today. We also need an API to create a "similar surface" of a given size and content-type; this would create a context and a source image object together.

We don't really care about API convenience in Gecko. Unlike a regular application, we don't draw specific content, we just provide abstractions. So for example we only paint gradients in a couple of places, we only paint text in a few places, etc. Therefore we don't need to worry about passing lots of parameters to functions.

We don't care about API stability.

=== Remoting ===

We need an API that can be remoted very efficiently, since we'll need it for sandboxed content processes that want to use D2D or whatever for drawing.

We probably need to assume from the start that system gfx libs can't be used in a sandboxed process.  (Correct this if it's wrong!)

Options for remoting seem to be
# Map API to something like the cairo-image-surface backend; i.e., don't remote anything except references to rendered shmem images.
# Map API to protocol designed exactly for remoting the API
# Map API to GL, re-use GL protocol we'll need for webgl
# Map API to backend-specific protocols

We'll likely want (1) for systems without a GPU, although we might regress on systems with very good SW libs (on which we could potentially use (2) or (4) to erase the regression).  To use D2D, we need either (2) or (4).  (3) is attractive for sharing more code.  (4) is unattractive for forking more code.

(cjones) Seems like the best solution is to support (1) (on some thread) and (2).

= Performance =

We need a thin wrapper around native backends, some of which use a lot of state in their contexts (Quartz), some of which don't (D2D).

cairo wraps a stateful context around mostly-stateless surface backends. This imposes unnecessary overhead on us (e.g. routing through the gstate) especially where the stateless surface backend maps back to a stateful platform abstraction (e.g. D2D clipping, Quartz everything).

For example, every time layout paints a display item, we want to explicitly set the clip region for the display item. Because cairo_t is stateful, we have to restore back to a known point, save, and set the clip (layout identifies when the clip region doesn't need to be changed to avoid this on every display item paint). But underneath cairo is creating clipper objects representing the "current clip" and passing them to a stateless surface backend. In turn, the backends for Quartz and D2D are managing clip state on the native contexts.

= Proposal =

=== API ===

Stateful abstractions are a pain because we reset them a lot anyway and mapping one stateful abstraction onto another is tricky where they don't quite match up (e.g. semantics of changing the transform while emitting a path).

So, go stateless ... almost. As noted above, we almost always need to account for a "current transform" and "current clip", so leave those in the context, otherwise we'd just have to pass them around everywhere, which would be annoying. Make everything else a parameter to the drawing calls.

It's already easy to manipulate transforms since we have SetMatrix as well as GetMatrix. Do the same for clipping; expose clip objects, GetClip/SetClip, and operations on clip objects. Allow the clip objects to have backend-private implementation.

No gstates or save/restore should be required, except of course we can have helper autos to save and restore matrix and/or clip.

Pack current operator, antialiasing mode, snapping mode, and sundry flags into a single flags word passed to each draw call.

Stateless means we use explicit path objects (backend-dependent implementation).

Provide explicit API for fillRect and other canvas operations so we don't have to waste time in backends identifying which fast path to use having gone through a generic path operation.

=== Implementation ===

For optimal efficiency with stateful backends we need to figure out what state needs to be updated at each drawing call and only update that state. For example, with Quartz, if the operator, antialiasing mode, clip and source pattern are the same as the previous drawing call, we shouldn't do Quartz calls to set them. Packing flags and modes into a single flags word will help quick identification of changes. We'll want to be able to identify reused clips and source patterns efficiently without making those objects too heavyweight. This may require some ingenuity.

=== Plan Of Attack ===

Start with canvas. We have performance needs there, the risk is lower and the implementation burden is lower.

* Implement new gfx API as a wrapper around cairo
* Clone nsCanvasRenderingContext2D with a run-time switch to use new implementation
* New Context2D uses new gfx API
** At this point we have something that works and we can iterate on the gfx API
* Implement new D2D and/or Quartz backends
** At this point we should have significant performance wins on real Web apps and something we want to ship
* Implement GL backend (lots more work)
* Extend API and backends with all the functionality we need
* Reimplement gfxContext on top of new gfx API (compile time switch probably needed, or possibly a branch)
* Incrementally convert all our graphics code to the new API

= Meeting Notes =

* Post-FF4, update cairo and plug in cairo-gl to canvas to see how it goes
* Don't make a big investment in improving cairo-gl
* Follow above plan
* Do easy things to make new-gfx reusable by other projects (simple STL, bool, namespaces) but not hard things (threads)
* Try GL backend on ANGLE before we commit to D3D backend(s)
* Options for improving software rendering
** New-API over Skia
** Improve cairo