Gecko:Layers

Layer API Proposals

Roc

// Reference counted, safe-for-cross-thread-use layer class.
// Conceptually it's just something that knows how to composite itself
// onto some parent surface/layer.
// Layers are immutable. They might have time-varying rendering (animation),
// but you can't modify one once it's created. This makes them easy to use
// safely across threads.
class Layer {
};

class FunctionOfTime<T> {
  // We assume that we can access the frame rate to figure out how many
  // samples we should provide
  static FunctionOfTime createSeries(TimeStamp start, TimeDuration duration,
                                     int numSamples, T* values);
};

// Generic superclass for helper object that creates a layer. This can only
// be used on one thread.
class LayerBuilder {
  // Indicate that the given layer is related to this one, e.g., the new layer
  // corresponds to the same element as the given layer.
  // We can use this to predict that the new layer will be used in the
  // same way as the given layer, for example, the eventual rendering
  // destination(s) of the new layer can be predicted to be whatever the old
  // layer was rendered to.
  void setAffinity(Layer);

  // Set rendering properties
  void setOpacity(float);
  // Set the transform used to render this layer onto the destination surface
  void setTransform(matrix);
  void setColorSpaceConversion(...);
  void setProgram(...);

  // Set animated rendering properties. The layer is still immutable from
  // the API point of view, but its properties will change over time.
  // Specifying a function over time here lets a compositing thread change
  // the properties without blocking on the main thread.
  void setAnimatedOpacity(FunctionOfTime<float>);
  void setAnimatedTransform(FunctionOfTime<matrix>);

  // Finish building and return the Layer. This can only be called once,
  // nothing else can be done with this LayerBuilder afterward.
  // In some cases this may return no layer, in particular when the builder
  // was created by ContainerLayerBuilder::addContainerChild/addRenderedChild
  // (the layer system may have rendered the child's contents directly into the
  // parent).
  Layer finish();
};

class YUVLayerBuilder : LayerBuilder {
  // Create a YUV layer with given size and format, and adopt the memory buffer
  YUVLayerBuilder(size, format, bufferToAdopt);
};

class WebGLBufferLayerBuilder : LayerBuilder {
  // Create a layer that's a logical copy (ideally copy on write) of the
  // underlying buffer managed by a WebGL canvas
  WebGLLayerBuilder(webGLBuffer);
};

class ContainerLayerBuilder : LayerBuilder {
  // format can be RGB, ARGB (eventually ARAGAB?).
  // This constructs a container layer that can be used anywhere.
  ContainerLayerBuilder(size, format);

  // The following methods can only be called after all LayerBuilder property
  // setters are done.

  // Add an existing layer
  addLayer(Layer);

  // Open a child container layer. This child must finish()
  // before another child can be added or this builder finishes.
  ContainerLayerBuilder addContainerChild(size, format);
  // Open a child rendered layer. This child must finish()
  // before another child can be added or this builder finishes.
  // RenderedLayers constructed this way may not need a temporary surface.
  RenderedLayerBuilder addRenderedChild(size, format);
};

class RenderedLayerBuilder : LayerBuilder {
  // format can be RGB, ARGB (eventually ARAGAB?)
  // This constructs a layer rendered via gfx that can be used anywhere
  // (and therefore requires a temporary surface).
  RenderedLayerBuilder(size, format);

  // create a (conceptual) copy of the given RenderedLayer so we can modify its
  // parameters or draw into it. The underlying buffer can be managed with
  // copy on write so if we don't ever call getContext, the buffer need not
  // be copied.
  RenderedLayerBuilder(Layer layer);

  // This can only be called after all LayerBuilder property setters are
  // done. The context cannot be used after finish() is called.
  gfxContext* getContext();
};

Add a method gfxContext::SetSource(Layer).

Add a way to return a Layer from a paint event (or just set it directly on the widget), so it gets rendered, possibly asynchronously on another thread.

Clients can use a mixture of retained Layers and recursive painting with each recursion level delimited by ContainerLayerBuilder::addContainerChild followed by finish() on the child.

The goal is to allow a pure cairo implementation of this API that's as efficient as we have today. In that implementation RenderedLayerBuilder::getContext tries to return a context that renders directly into the underlying surface for some ancestor. Of course we also want to have a GL or D3D implementation that's fast, but will require more temporary surfaces if we're not using cairo-gl.

When we go to off-main-thread compositing we'll want to add support for animation and other stuff. For example we might want a YUVSeriesLayerBuilder that can select from a queue of timestamped frames based on the current time. The rendering property setters on LayerBuilder would be extended with animating setters that take a list of timestamped values, or perhaps the parameters of actual transition functions.

Bas

// The controlling class that controls the composition of frames. This
// lives on a rectangular area on the client's screen, and controls the
// composition of all layers on the compositor. This runs its own thread
// internally from which all OpenGL/D3D operations are executed. All re-
// scheduling of drawing and invalidations are run based on operations
// executed on the compositor and its layers.
class Compositor
{
  // Create a layer that can be used to render to, the size here
  // describes the size in pixels. The format the format of the data,
  // This can be RGB, RGBA, YUV. The compositor will know what to do
  // with these layers, and how to render them properly. When the last
  // reference to the layer dies there will be only one left, and it's
  // ready to be destroyed. Type can be one of hardware or managed.
  // Only managed layers can be drawn to directly from software.
  // Any created layer can contain other layers inside, places anywhere
  // on its surface.
  Layer *CreateLayer(size, format, type);
};

// These are operations that can be executed on all layers.
class ILayer
{
  // Color by which the layers pixels are multiplied,
  // This contains an alpha value so opacity can implicitly
  // be controlled.
  SetColor(color); 

  // Sets an affine transformation to place the layer with.
  SetTransform(matrix);

  // Add a layer to this layer. This layer may be blitted onto
  // this layer's hardware surface.
  AddLayer(ILayer);

  // Optional pixel shader program to run on this layer. This can be
  // used to apply a variety of effects to the layer when rendered.
  SetShader(shader);
};

// Layers exposing this interface allow access to the surface. Double
// buffered, this means that if it's currently being drawn to the compositor
// will simply draw the texture. This will ensure rendering of the compositor
// area doesn't stall waiting on an expensive software render.
class ILockableLayer
{
  // Lock the surface of this layer. Returns a gfxContext to draw to.
  gfxContext *Lock();

  // Unlock the surface, this means we're done. And will signal the
  // compositor to update the associated texture and redraw.
  Unlock();
};

// Layers exposing this interface can have their hardware surface accessed,
// which can then be used as a render target for other accelerated parts of
// the code.
class IHardwareLayer
{
  // Return hardware surface in whatever structure we pick. Might need
  // locking/unlocking logic.
  HardwareSurface *Surface();
};

// This class controls animations on objects, any class can be made to
// implement it, but we'd most likely provide some standard implementations.
// Any state it wants to maintain is contained on an implementation level.
class IAnimator
{
  // Called by the compositor when starting a rendering cycle, with
  // the elapsed time.
  virtual void AdvanceTime(double aTime);

  // This assigns the animator to a frame and registers with its compositor.
  void Assign(ILayer *aLayer);
}