NPAPI:AsyncDrawing

Status

Under consideration.

Contributors

• Last modified: April 28, 2011
• Authors: Bas Schouten (Mozilla Corporation), Josh Aas (Mozilla Corporation)
• Contributors: Robert O'Callahan (Mozilla Corporation)
• QA contact: George Carstoiu (irc: GeorgeCarstoiu)

Overview

This specification allows plugins to draw asynchronously to surfaces that are potentially located in video memory.

Negotiating Async Drawing

For documentation on negotiating drawing models, see NPAPI:Models. This specification covers all async models, which use the same drawing mechanics with platform-specific data structures. The following models are currently specified:

• NPDrawingModelAsyncBitmapSurface (NPDrawingModel = TBA)
• NPDrawingModelAsyncWindowsDXGISurface (NPDrawingModel = TBA)
• NPDrawingModelAsyncWindowsDX9ExSurface (NPDrawingModel = TBA)

Support query variables are:

• NPNVsupportsAsyncBitmapSurfaceBool (NPNVariable = TBA)
• NPNVsupportsAsyncWindowsDXGISurfaceBool (NPNVariable = TBA)
• NPNVsupportsAsyncWindowsDX9ExSurfaceBool (NPNVariable = TBA)

Since these are the first new models for Windows and Linux the existing models will be assigned names:

• NPDrawingModelSyncWin (NPDrawingModel = TBA)
• NPDrawingModelSyncX (NPDrawingModel = TBA)

General Async Drawing Mechanics

Any async drawing model will allow plugins to create one or more surfaces for drawing. The plugin will tell the host which surface is current at any given time and the host will display the current plugin surface for as long as it is current. While a surface is current the plugin is not allowed to modify it. This system allows the plugin to choose whether to double or triple buffer.

Surfaces are specified via the NPAsyncSurface structure. This structure contains a pointer to a platform-specific resource of the type defined by the drawing model:

  /* These formats describe the format in the memory byte-order. This means if
   * a 32-bit value of a pixel is viewed on a little-endian system the layout will
   * be 0xAARRGGBB. Since the Alpha channel will be stored in the most significant
   * bits */
  typedef enum {
    /* 32-bit per pixel 8-bit per channel - premultiplied alpha */
    NPImageFormatBGRA32     = 0x1,
    /* 32-bit per pixel 8-bit per channel - 1 unused channel */
    NPImageFormatBGRX32     = 0x2
  } NPImageFormat;
 
  /* This is version 0 of the structure. */
  typedef struct _NPAsyncSurface
  {
    uint32_t version;
    NPSize size;
    NPImageFormat format;
    union {
      struct { unsigned char *data; uint32_t stride; };
      #ifdef XP_WIN
      HANDLE sharedHandle
      #endif
    };
  } NPAsyncSurface;

New surfaces can be initialized via a new function called NPN_InitAsyncSurface:

  NPError NPN_InitAsyncSurface(NPP instance, NPSize* size,
                               int32_t format, void* initData,
                               NPSurface* surface);

When a plugin is done with a surface the surface should be finalized with NPN_FinalizeAsyncSurface:

  NPError NPN_FinalizeAsyncSurface(NPP instance, NPAsyncSurface* surface);

What exactly this means depends on the drawing model's ownership policy and the implementation. If the finalized surface is current, its contents will be drawn until another surface is set current or the plugin goes away.

The currently displayed surface can be set via a new function called NPN_SetCurrentAsyncSurface:

  NPError NPN_SetCurrentAsyncSurface(NPP instance, NPAsyncSurface* surface);

Surfaces cannot be modified while they are current.

All three functions (init, finalize, set current) can be called from any thread.

NPDrawingModelAsyncBitmapSurface

This model should be available on all platforms.

Plugins should create a bitmap surface by calling NPN_InitAsyncSurface with a NULL value for initData. If an NPImageFormat is not supported then surface creation will fail.

Plugins should destroy bitmap surfaces by calling NPN_FinalizeAsyncSurface. This will allow the browser to release the bitmap memory.

NPDrawingModelAsyncWindowsSharedSurface

This drawing model will only be valid on Windows Vista and higher in order to simplify hardware accelerated surface sharing.

Plugins should create a Windows shared surface by calling NPN_InitAsyncSurface with a NULL value for initData. The resulting NPAsyncSurface will point to a HANDLE that can be used, for example, through OpenSharedResource in order to create a texture for the user. In order to allow fast drawing to any hardware surfaces, the host will acquire the handle from IDXGISurface::GetSharedHandle. This shared handle will represent a texture which is usable as a render target and is valid as a shader resource. The plugin can open this shared handle as a texture and then use it as a render target for any drawing operations.

When a surface is set as current the plugin is responsible for making sure all drawing calls on that surface have completed execution!

Plugins should finalize Windows shared surfaces by calling NPN_FinalizeAsyncSurface when they are done with the surface.

This sample code illustrates usage of this drawing model:

ID3D10Device *pDevice10;
// Initialize device.

NPAsyncSurface *npFrontBuffer = new NPAsyncSurface;
NPAsyncSurface *npBackBuffer = new NPAsyncSurface;
ID3D10Texture2D *frontBuffer;
ID3D10Texture2D *backBuffer;

npFrontBuffer->size.width = npBackBuffer->size.width = pluginwidth;
npFrontBuffer->size.height = npBackBuffer->size.height = pluginheight;
npFrontBuffer->format = npBackBuffer->format = NPImageFormatXRGB32;

NPN_InitAsyncSurface(instance, npFrontBuffer);
NPN_InitAsyncSurface(instance, npBackBuffer);

pDevice10->OpenSharedResource(npFrontBuffer->handle,
                              __uuidof(ID3D10Texture2D),
                              (void**)(&frontBuffer));
pDevice10->OpenSharedResource(npBackBuffer->handle,
                              __uuidof(ID3D10Texture2D),
                              (void**)(&backBuffer));

while (painting) {
  // Draw to backBuffer texture
  NPN_SetCurrentAsyncSurface(instance, npBackBuffer);
  ID3D10Texture2D *tmp = frontBuffer;
  NPAsyncSurface *npTmp = npFrontBuffer;
  frontBuffer = backBuffer;
  npFrontBuffer = npBackBuffer;
  backBuffer = tmp;
  npBackBuffer = npTmp;
  pDevice10->Flush();
}

frontBuffer->Release();
backBuffer->Release();

NPN_FinalizeAsyncSurface(instance, npFrontBuffer);
NPN_FinalizeAsyncSurface(instance, npBackBuffer);

delete npFrontBuffer;
delete npBackBuffer;

NPDrawingModelAsyncWindowsDX9ExSurface

This drawing model will only be valid on Windows Vista and higher in order to simplify hardware accelerated surface sharing for plugins wanting to use Direct3D 9Ex.

In this drawing model the plugin will be responsible for actual creation of the texture. The reason for this behavior is that textures created in Direct3D 10 cannot be shared using Direct3D 9/10 interoperability. Textures created with Direct3D 9 can be opened in Direct3D 10. In this model the plugin will create a texture using the CreateTexture call on a Direct3D 9Ex device. The pSharedHandle argument will be used to acquire a shared handle for that texture. This shared handle should be set as the data member of the NPAsyncSurface structure before NPN_InitAsyncSurface is called to initialize the surface. The plugin must also set the stride member of NPAsyncSurface before calling NPN_InitAsyncSurface.

The return value of the NPN_InitAsyncSurface call will indicate if the surface was successfully accessed by the host. When a surface is set current the plugin must make sure all drawing calls to that surface have finished execution. For Direct3D 9Ex this means making sure that the GPU has processed all graphics command queued to draw to that surface in order to prevent race condition with a host using another device. One way of doing this is do use a D3DQUERYTYPE_EVENT query to determine when the graphics pipeline has finished executing all events. Another method is to execute a blocking readback from a small (i.e. 1x1 pixel) portion of the texture. When the readback has successfully executed then the GPU has finished the drawing calls to the surface. Failure to follow these guidelines may result in incomplete drawing of the contents of the texture.

Plugins should finalize surfaces by calling NPN_FinalizeAsyncSurface.

(Code sample coming soon)

Open Issues

• Add sample code for NPDrawingModelAsyncWindowsDX9ExSurface
• Add accelerated Linux drawing model.