NPAPI:AsyncDrawing

Status

Under consideration.

Contributors

• Last modified: April 20, 2011
• Authors: Bas Schouten (Mozilla Corporation), Josh Aas (Mozilla Corporation)
• Contributors: Robert O'Callahan (Mozilla Corporation)

Overview

This specification allows plugins to draw asynchronously to surfaces that are potentially located in video memory.

Negotiating Async Drawing

For documentation on negotiating drawing models, see NPAPI:Models. This specification covers all async models, which use the same drawing mechanics with platform-specific data structures. The following models are currently specified:

• NPDrawingModelAsyncBitmapSurface (NPDrawingModel = TBA)
• NPDrawingModelAsyncWindowsDXGISurface (NPDrawingModel = TBA)
• NPDrawingModelAsyncWindowsDX9ExSurface (NPDrawingModel = TBA)

Support query variables are:

• NPNVsupportsAsyncBitmapSurfaceBool (NPNVariable = TBA)
• NPNVsupportsAsyncWindowsDXGISurfaceBool (NPNVariable = TBA)
• NPNVsupportsAsyncWindowsDX9ExSurfaceBool (NPNVariable = TBA)

Async Drawing Mechanics

Any async drawing model will allow plugins to create one or more surfaces for drawing. The plugin will tell the host which surface is current at any given time and the host will display the current plugin surface for as long as it is current. While a surface is current the plugin is not allowed to modify it. This system allows the plugin to choose whether to double or triple buffer.

Surfaces are specified via the NPAsyncSurface structure. This structure contains a pointer to a platform-specific resource of the type defined by the drawing model:

  /* "P" suffix means pre-multiplied alpha. */
  typedef enum {
    NPImageFormatARGB32P    = 0x1,
    NPImageFormatARGB32     = 0x2,
    NPImageFormatXRGB32     = 0x4
  } NPImageFormat;
 
  typedef struct _NPAsyncSurface
  {
    uint32_t version;
    NPSize size;
    NPImageFormat format;
    uint32_t      stride;
    void *data;
  } NPAsyncSurface;

New surfaces can be initialized via a new function called NPN_InitAsyncSurface:

  NPError NPN_InitAsyncSurface(NPP instance, NPAsyncSurface* surface);

Plugins are always responsible for initializing the version, size, and format members in an NPAsyncSurface before calling NPN_InitAsyncSurface with it. If there is only one version of the surface then plugins should set the version member to 0. The stride member may be initialized by the plugin or the browser depending on the selected drawing model. Whether or not data is initialized by the plugin or the browser depends on the selected drawing model but upon return from NPN_InitAsyncSurface the data pointer should point to valid surface data. The value of data will be NULL if an error occurred.

When a plugin is done with a surface the surface should be finalized with NPN_FinalizeAsyncSurface:

  NPError NPN_FinalizeAsyncSurface(NPP instance, NPAsyncSurface* surface);

What exactly this means depends on the drawing model's ownership policy and the implementation. If the finalized surface is current, its contents will be drawn until another surface is set current or the plugin goes away.

The currently displayed surface can be set via a new function called NPN_SetCurrentAsyncSurface:

  NPError NPN_SetCurrentAsyncSurface(NPP instance, NPAsyncSurface* surface);

Surfaces cannot be modified while they are current.

All three functions (init, finalize, set current) can be called from any thread.

NPDrawingModelAsyncBitmapSurface

This model should be available on all platforms.

On succesful return the data pointer of the NPAsyncSurface will point to the data in memory for the surface. The stride member will reflect the stride of the returned image surface.

If an NPImageFormat is not supported then surface creation will fail.

NPDrawingModelAsyncWindowsSharedSurface

This drawing model will only be valid on Windows Vista and higher in order to simplify hardware accelerated surface sharing.

The data pointer of the NPAsyncSurface will be a HANDLE that can be used, for example, through OpenSharedResource in order to create a texture for the user. In order to allow fast drawing to any hardware surfaces the host will acquire the handle from IDXGISurface::GetSharedHandle. This shared handle will represent a texture which is usable as a render target and is valid as a shader resource. The plugin can open this shared handle as a texture and then use it as a render target for any drawing operations.

When a surface is set as current the plugin is responsible for making sure all drawing calls on that surface have completed execution!

This sample code illustrates usage of this drawing model:

ID3D10Device *pDevice10;
// Initialize device.

NPAsyncSurface *npFrontBuffer = new NPAsyncSurface;
NPAsyncSurface *npBackBuffer = new NPAsyncSurface;
ID3D10Texture2D *frontBuffer;
ID3D10Texture2D *backBuffer;

npFrontBuffer->size.width = npBackBuffer->size.width = pluginwidth;
npFrontBuffer->size.height = npBackBuffer->size.height = pluginheight;
npFrontBuffer->format = npBackBuffer->format = NPImageFormatXRGB32;

NPN_CreateAsyncSurface(instance, npFrontBuffer);
NPN_CreateAsyncSurface(instance, npBackBuffer);

pDevice10->OpenSharedResource(npFrontBuffer->handle,
                              __uuidof(ID3D10Texture2D),
                              (void**)(&frontBuffer));
pDevice10->OpenSharedResource(npBackBuffer->handle,
                              __uuidof(ID3D10Texture2D),
                              (void**)(&backBuffer));

while (painting) {
  // Draw to backBuffer texture
  NPN_SetCurrentAsyncSurface(instance, npBackBuffer);
  ID3D10Texture2D *tmp = frontBuffer;
  NPAsyncSurface *npTmp = npFrontBuffer;
  frontBuffer = backBuffer;
  npFrontBuffer = npBackBuffer;
  backBuffer = tmp;
  npBackBuffer = npTmp;
  pDevice10->Flush();
}

frontBuffer->Release();
backBuffer->Release();

NPN_DestroyAsyncSurface(instance, npFrontBuffer);
NPN_DestroyAsyncSurface(instance, npBackBuffer);

delete npFrontBuffer;
delete npBackBuffer;

NPDrawingModelAsyncWindowsDX9ExSurface

This drawing model will only be valid on Windows Vista and higher in order to simplify hardware accelerated surface sharing for plugins wanting to use Direct3D 9Ex.

In this drawing model the plugin will be responsible for actual creation of the texture. The reason for this behavior is that textures created in Direct3D 10 cannot be shared using Direct3D 9/10 interop. Textures created with Direct3D 9 can be opened in Direct3D 10. In this model the plugin will create a texture using the CreateTexture call on a Direct3D 9Ex device. The pSharedHandle argument will be used to acquire a shared handle for that texture. This shared handle will then, along with all the other information, be passed into the NPAsyncSurface structure, with the data member set to the shared handle.

The return value of the NPN_CreateAsyncSurface call will indicate if the surface was successfully accessed by the host. When a surface is set current the plugin must make sure all drawing calls to that surface have finished execution. For Direct3D 9Ex this means making sure that the GPU has processed all graphics command queued to draw to that surface in order to prevent race condition with a host using another device. One way of doing this is do use a D3DQUERYTYPE_EVENT query to determine when the graphics pipeline has finished executing all events. Another method is to execute a blocking readback from a small (i.e. 1x1 pixel) portion of the texture, when the readback has successfully executed the GPU has finished the drawing calls to the surface. Failure to follow these guidelines may result in incomplete drawing of the contents of the texture.

Code samples will follow.

Open Issues

• Add accelerated Linux drawing model.