MozillaWiki

NPAPI:AsyncDrawing: Difference between revisions

Page Discussion

NPAPI:AsyncDrawing (view source)

Revision as of 06:47, 15 November 2015

1,689 bytes added ,  15 November 2015
m
no edit summary
No edit summary
mNo edit summary
 
(37 intermediate revisions by 6 users not shown)
Line 1: Line 1:
= Status =
= Status =


Under consideration.
Accepted, ready for implementation.  


= Contributors =
= Contributors =


* Last modified: April 28, 2011
* Last modified: April 19, 2012
* Authors: Bas Schouten (Mozilla Corporation), Josh Aas (Mozilla Corporation)
* Authors: Bas Schouten (Mozilla Corporation), Josh Aas (Mozilla Corporation)
* Contributors: Robert O'Callahan (Mozilla Corporation)
* Contributors: Robert O'Callahan (Mozilla Corporation)
Line 17: Line 17:
For documentation on negotiating drawing models, see [[NPAPI:Models]]. This specification covers all async models, which use the same drawing mechanics with platform-specific data structures. The following models are currently specified:
For documentation on negotiating drawing models, see [[NPAPI:Models]]. This specification covers all async models, which use the same drawing mechanics with platform-specific data structures. The following models are currently specified:


* NPDrawingModelAsyncBitmapSurface (NPDrawingModel = TBA)
* NPDrawingModelAsyncBitmapSurface (NPDrawingModel = 7)
* NPDrawingModelAsyncWindowsDXGISurface (NPDrawingModel = TBA)
* NPDrawingModelAsyncWindowsDXGISurface (NPDrawingModel = 8)
* NPDrawingModelAsyncWindowsDX9ExSurface (NPDrawingModel = TBA)


Support query variables are:
Support query variables are:


* NPNVsupportsAsyncBitmapSurfaceBool (NPNVariable = TBA)
* NPNVsupportsAsyncBitmapSurfaceBool (NPNVariable = 2007)
* NPNVsupportsAsyncWindowsDXGISurfaceBool (NPNVariable = TBA)
* NPNVsupportsAsyncWindowsDXGISurfaceBool (NPNVariable = 2008)
* NPNVsupportsAsyncWindowsDX9ExSurfaceBool (NPNVariable = TBA)
* NPNVpreferredDXGIAdapter (NPNVariable = 2009)


Since these are the first new models for Windows and Linux the existing models will be assigned names:
Since these are the first new models for Windows and Linux the existing models will be assigned names:


* NPDrawingModelSyncWin (NPDrawingModel = TBA)
* NPDrawingModelSyncWin (NPDrawingModel = 5)
* NPDrawingModelSyncX (NPDrawingModel = TBA)
* NPDrawingModelSyncX (NPDrawingModel = 6)


= General Async Drawing Mechanics =
= General Async Drawing Mechanics =
Line 42: Line 41:
   /* These formats describe the format in the memory byte-order. This means if
   /* These formats describe the format in the memory byte-order. This means if
   * a 32-bit value of a pixel is viewed on a little-endian system the layout will
   * a 32-bit value of a pixel is viewed on a little-endian system the layout will
   * be 0xAARRGGBB. Since the Alpha channel will be stored in the most significant
   * be 0xAARRGGBB. The Alpha channel will be stored in the most significant
   * bits */
   * bits. */
   typedef enum {
   typedef enum {
     /* 32-bit per pixel 8-bit per channel - premultiplied alpha */
     /* 32-bit per pixel 8-bit per channel - premultiplied alpha */
Line 51: Line 50:
   } NPImageFormat;
   } NPImageFormat;
   
   
  /* This is version 0 of the structure. */
   typedef struct _NPAsyncSurface
   typedef struct _NPAsyncSurface
   {
   {
Line 57: Line 57:
     NPImageFormat format;
     NPImageFormat format;
     union {
     union {
       struct { unsigned char *data; uint32_t stride; };
       struct {
        unsigned char *data;
        uint32_t stride;
      } bitmap;
       #ifdef XP_WIN
       #ifdef XP_WIN
       HANDLE sharedHandle
       HANDLE sharedHandle
Line 79: Line 82:
</pre>
</pre>


What exactly this means depends on the drawing model's ownership policy and the implementation. If the finalized surface is current, its contents will be drawn until another surface is set current or the plugin goes away.
What exactly this means depends on the drawing model's ownership policy and the implementation. If the finalized surface is current, this function will return an error.


The currently displayed surface can be set via a new function called <code>NPN_SetCurrentAsyncSurface</code>:
The currently displayed surface can be set via a new function called <code>NPN_SetCurrentAsyncSurface</code>:


<pre>
<pre>
   NPError NPN_SetCurrentAsyncSurface(NPP instance, NPAsyncSurface* surface);
   void NPN_SetCurrentAsyncSurface(NPP instance, NPAsyncSurface* surface, NPRect *changed);
</pre>
</pre>


Surfaces cannot be modified while they are current.
Surfaces cannot be modified while they are current. A modified rectangle may be provided in the <code>changed</code> argument to give the host a hint on which part of the surface was modified with respect to the previous current surface so that the browser can choose to optimize the composition process. There is no guarantee only this rectangle will be recomposed though and the entire surface needs to be valid.


All three functions (init, finalize, set current) can be called from any thread.
SetCurrent can take NULL as pthe surface. This removes the previous current surface. The plugin will render nothing. This should be used before finalizing the last surface. All three functions must be called on the plugin thread.
 
Reentrant calls from the browser into the plugin do not occur during a SetCurrent invocation from the main thread. The browser is expected to implement SetCurrent using a locking protocol that cannot block for long. For example, SetCurrent might block only while the previous surface is being read by the browser's compositing.
 
= Throttling Painting =
 
To enable the plugin to match its frame rate to the browser's, the browser notifies the plugin each time it has composited a frame that used the plugin's current surface. This notification fires even if the plugin's current surface is NULL.
 
<pre>
  void NPP_DidComposite(NPP instance);
</pre>
 
== Hidden Plugins ==
 
While a plugin is in a hidden tab (or is hidden for some other reason), the browser doesn't have to fire NPP_DidComposite. When the plugin becomes visible, the browser won't wait for the plugin to re-render before compositing --- that would defeat the purpose of an asynchronous API. This could mean displaying an old, stale frame of the plugin for a moment. A plugin can mitigate this by calling SetCurrentAsyncSurface at a low rate even while DidComposite is not being received. In some situations it might be appropriate for a plugin to set the current surface to NULL while it's hidden (NPP_SetWindow having set an empty clip-rect); this would cause the plugin to render nothing instead of a stale frame when it becomes visible.


= NPDrawingModelAsyncBitmapSurface =
= NPDrawingModelAsyncBitmapSurface =
Line 103: Line 120:
This drawing model will only be valid on Windows Vista and higher in order to simplify hardware accelerated surface sharing.
This drawing model will only be valid on Windows Vista and higher in order to simplify hardware accelerated surface sharing.


Plugins should create a Windows shared surface by calling <code>NPN_InitAsyncSurface</code> with a <code>NULL</code> value for <code>initData</code>. The resulting <code>NPAsyncSurface</code> will point to a <code>HANDLE</code> that can be used, for example, through [http://msdn.microsoft.com/en-us/library/bb173598%28v=VS.85%29.aspx OpenSharedResource] in order to create a texture for the user. In order to allow fast drawing to any hardware surfaces, the host will acquire the handle from [http://msdn.microsoft.com/en-us/library/bb174562%28v=VS.85%29.aspx IDXGISurface::GetSharedHandle]. This shared handle will represent a texture which is usable as a render target and is valid as a shader resource. The plugin can open this shared handle as a texture and then use it as a render target for any drawing operations.
Plugins should create a Windows shared surface by calling <code>NPN_InitAsyncSurface</code> with a <code>NULL</code> value for <code>initData</code>. The resulting <code>NPAsyncSurface</code> will point to a <code>HANDLE</code> that can be used, for example, through [http://msdn.microsoft.com/en-us/library/bb173598%28v=VS.85%29.aspx OpenSharedResource] in order to create a texture for the user. In order to allow fast drawing to any hardware surfaces, the host will acquire the handle from [http://msdn.microsoft.com/en-us/library/bb174562%28v=VS.85%29.aspx IDXGISurface::GetSharedHandle]. This shared handle will represent a texture which is usable as a render target and is valid as a shader resource. The plugin can open this shared handle as a texture and then use it as a render target for any drawing operations. In order to synchronize GPU access to the surface the resource created will support the [http://msdn.microsoft.com/en-us/library/windows/desktop/ff471339%28v=vs.85%29.aspx IDXGIKeyedMutex] interface, the key used for both acquiring and releasing sync is 0.


When a surface is set as current the plugin is responsible for making sure all drawing calls on that surface have completed execution!
Plugins should finalize Windows shared surfaces by calling <code>NPN_FinalizeAsyncSurface</code> when they are done with the surface.


Plugins should finalize Windows shared surfaces by calling <code>NPN_FinalizeAsyncSurface</code> when they are done with the surface.
== Creating the Device ==
 
On dual-GPU systems, the browser and plugin process may unintentionally bind to different GPUs if default DXGI parameters are used. When this happens the plugin's draw calls will most likely fail, or crash. To address this, it is necessary to open the browser's shared DXGI surfaces on the same adapter. The adapter can be found using <tt>NPNVpreferredDXGIAdapter</tt>. For example,
 
<pre>
static IDXGIAdapter1*
FindDXGIAdapter(NPP npp, IDXGIFactory1* factory)
{
  DXGI_ADAPTER_DESC preferred;
  if (NPN_GetValue(npp, NPNVpreferredDXGIAdapter, &preferred) != NPERR_NO_ERROR) {
    return NULL;
  }
 
  for (UINT index = 0; ; index++) {
    IDXGIAdapter* adapter = NULL;
    if (FAILED(factory1->EnumAdapters1(index, &adapter)) || !adapter) {
      return nullptr;
    }
 
    DXGI_ADAPTER_DESC desc;
    if (SUCCEEDED(adapter->GetDesc(&desc)) &&
        desc.AdapterLuid.LowPart == preferred.AdapterLuid.LowPart &&
        desc.AdapterLuid.HighPart == preferred.AdapterLuid.HighPart)
    {
      return adapter;
    }
    adapter->Release();
  }
  return NULL;
}
</pre>
 
The adapter returned by this function can then be used with any DXGI device creation function, such as <tt>D3D10CreateDevice</tt>. Failure to negotiate the adapter properly (or failure to use an IDXGIAdapter1) will result in undefined behavior.
 
If no matching adapter can be found, the safest course of action for the plugin is to switch to a different drawing model (such the AsyncBitmap model).
 
== Sample Code ==


This sample code illustrates usage of this drawing model:
This sample code illustrates usage of this drawing model:


<pre>
  ID3D10Device *pDevice10;
  ID3D10Device *pDevice10;
// Initialize device.
  NPAsyncSurface *npFrontBuffer = new NPAsyncSurface;
  NPAsyncSurface *npFrontBuffer = new NPAsyncSurface;
  NPAsyncSurface *npBackBuffer = new NPAsyncSurface;
  NPAsyncSurface *npBackBuffer = new NPAsyncSurface;
Line 119: Line 171:
  ID3D10Texture2D *backBuffer;
  ID3D10Texture2D *backBuffer;
   
   
  npFrontBuffer->size.width = npBackBuffer->size.width = pluginwidth;
  void Init() {
  npFrontBuffer->size.height = npBackBuffer->size.height = pluginheight;
  ... Initialize pDevice10 ...
npFrontBuffer->format = npBackBuffer->format = NPImageFormatXRGB32;
  NPSize size;
  size.width = pluginwidth;
  size.height = pluginheight;
   
  NPN_InitAsyncSurface(instance, size, NPImageFormatBGRX32, NULL, npFrontBuffer);
  NPN_InitAsyncSurface(instance, size, NPImageFormatBGRX32, NULL, npBackBuffer);
  pDevice10->OpenSharedResource(npFrontBuffer->handle,
                                __uuidof(ID3D10Texture2D),
                                (void**)(&frontBuffer));
  pDevice10->OpenSharedResource(npBackBuffer->handle,
                                __uuidof(ID3D10Texture2D),
                                (void**)(&backBuffer));
  DrawFrame();
}
   
   
  NPN_InitAsyncSurface(instance, npFrontBuffer);
  void DrawFrame() {
NPN_InitAsyncSurface(instance, npBackBuffer);
  IDXGIKeyedMutex *mutex;
  backBuffer->QueryInterface(&mutex);
  mutex->AcquireSync(0);
   
   
pDevice10->OpenSharedResource(npFrontBuffer->handle,
  ... Draw to backBuffer texture ...
                              __uuidof(ID3D10Texture2D),
                              (void**)(&frontBuffer));
pDevice10->OpenSharedResource(npBackBuffer->handle,
                              __uuidof(ID3D10Texture2D),
                              (void**)(&backBuffer));
   
   
while (painting) {
  mutex->ReleaseSync(0);
   // Draw to backBuffer texture
   mutex->Release();
 
   NPN_SetCurrentAsyncSurface(instance, npBackBuffer);
   NPN_SetCurrentAsyncSurface(instance, npBackBuffer);
   ID3D10Texture2D *tmp = frontBuffer;
   ID3D10Texture2D *tmp = frontBuffer;
Line 142: Line 207:
   backBuffer = tmp;
   backBuffer = tmp;
   npBackBuffer = npTmp;
   npBackBuffer = npTmp;
  pDevice10->Flush();
  }
  }
   
   
  frontBuffer->Release();
  void NPP_DidComposite() {
backBuffer->Release();
  DrawFrame();
}
   
   
  NPN_FinalizeAsyncSurface(instance, npFrontBuffer);
  void Shutdown() {
NPN_FinalizeAsyncSurface(instance, npBackBuffer);
  frontBuffer->Release();
  backBuffer->Release();
   
   
  delete npFrontBuffer;
  NPN_SetCurrentAsyncSurface(instance, NULL);
delete npBackBuffer;
  NPN_FinalizeAsyncSurface(instance, npFrontBuffer);
 
  NPN_FinalizeAsyncSurface(instance, npBackBuffer);
= NPDrawingModelAsyncWindowsDX9ExSurface =
   
 
  delete npFrontBuffer;
This drawing model will only be valid on Windows Vista and higher in order to simplify hardware accelerated surface sharing for plugins wanting to use Direct3D 9Ex.
  delete npBackBuffer;
 
}
In this drawing model the plugin will be responsible for actual creation of the texture. The reason for this behavior is that textures created in Direct3D 10 cannot be shared using Direct3D 9/10 interoperability. Textures created with Direct3D 9 can be opened in Direct3D 10. In this model the plugin will create a texture using the <code>CreateTexture</code> call on a Direct3D 9Ex device. The <code>pSharedHandle</code> argument will be used to acquire a shared handle for that texture. This shared handle should be set as the <code>data</code> member of the <code>NPAsyncSurface</code> structure before <code>NPN_InitAsyncSurface</code> is called to initialize the surface. The plugin must also set the <code>stride</code> member of <code>NPAsyncSurface</code> before calling <code>NPN_InitAsyncSurface</code>.
</pre>
 
The return value of the <code>NPN_InitAsyncSurface</code> call will indicate if the surface was successfully accessed by the host. When a surface is set current the plugin must make sure all drawing calls to that surface have finished execution. For Direct3D 9Ex this means making sure that the GPU has processed all graphics command queued to draw to that surface in order to prevent race condition with a host using another device. One way of doing this is do use a <code>D3DQUERYTYPE_EVENT</code> query to determine when the graphics pipeline has finished executing all events. Another method is to execute a blocking readback from a small (i.e. 1x1 pixel) portion of the texture. When the readback has successfully executed then the GPU has finished the drawing calls to the surface. Failure to follow these guidelines may result in incomplete drawing of the contents of the texture.
 
Plugins should finalize surfaces by calling <code>NPN_FinalizeAsyncSurface</code>.
 
(Code sample coming soon)
 
= Open Issues =
 
* Add accelerated Linux drawing model.
Dvander
Confirmed users
156

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/307096...1105676"