MozillaWiki

NPAPI:AsyncDrawing: Difference between revisions

Page Discussion

NPAPI:AsyncDrawing (view source)

Revision as of 06:47, 15 November 2015

975 bytes removed ,  15 November 2015
m
no edit summary
mNo edit summary
 
(19 intermediate revisions by 4 users not shown)
Line 1: Line 1:
= Status =
= Status =


Under consideration.
Accepted, ready for implementation.  


= Contributors =
= Contributors =


* Last modified: November 24, 2011
* Last modified: April 19, 2012
* Authors: Bas Schouten (Mozilla Corporation), Josh Aas (Mozilla Corporation)
* Authors: Bas Schouten (Mozilla Corporation), Josh Aas (Mozilla Corporation)
* Contributors: Robert O'Callahan (Mozilla Corporation)
* Contributors: Robert O'Callahan (Mozilla Corporation)
Line 17: Line 17:
For documentation on negotiating drawing models, see [[NPAPI:Models]]. This specification covers all async models, which use the same drawing mechanics with platform-specific data structures. The following models are currently specified:
For documentation on negotiating drawing models, see [[NPAPI:Models]]. This specification covers all async models, which use the same drawing mechanics with platform-specific data structures. The following models are currently specified:


* NPDrawingModelAsyncBitmapSurface (NPDrawingModel = TBA)
* NPDrawingModelAsyncBitmapSurface (NPDrawingModel = 7)
* NPDrawingModelAsyncWindowsDXGISurface (NPDrawingModel = TBA)
* NPDrawingModelAsyncWindowsDXGISurface (NPDrawingModel = 8)
* NPDrawingModelAsyncWindowsDX9ExSurface (NPDrawingModel = TBA)


Support query variables are:
Support query variables are:


* NPNVsupportsAsyncBitmapSurfaceBool (NPNVariable = TBA)
* NPNVsupportsAsyncBitmapSurfaceBool (NPNVariable = 2007)
* NPNVsupportsAsyncWindowsDXGISurfaceBool (NPNVariable = TBA)
* NPNVsupportsAsyncWindowsDXGISurfaceBool (NPNVariable = 2008)
* NPNVsupportsAsyncWindowsDX9ExSurfaceBool (NPNVariable = TBA)
* NPNVpreferredDXGIAdapter (NPNVariable = 2009)


Since these are the first new models for Windows and Linux the existing models will be assigned names:
Since these are the first new models for Windows and Linux the existing models will be assigned names:


* NPDrawingModelSyncWin (NPDrawingModel = TBA)
* NPDrawingModelSyncWin (NPDrawingModel = 5)
* NPDrawingModelSyncX (NPDrawingModel = TBA)
* NPDrawingModelSyncX (NPDrawingModel = 6)


= General Async Drawing Mechanics =
= General Async Drawing Mechanics =
Line 58: Line 57:
     NPImageFormat format;
     NPImageFormat format;
     union {
     union {
       struct { unsigned char *data; uint32_t stride; };
       struct {
        unsigned char *data;
        uint32_t stride;
      } bitmap;
       #ifdef XP_WIN
       #ifdef XP_WIN
       HANDLE sharedHandle
       HANDLE sharedHandle
Line 90: Line 92:
Surfaces cannot be modified while they are current. A modified rectangle may be provided in the <code>changed</code> argument to give the host a hint on which part of the surface was modified with respect to the previous current surface so that the browser can choose to optimize the composition process. There is no guarantee only this rectangle will be recomposed though and the entire surface needs to be valid.
Surfaces cannot be modified while they are current. A modified rectangle may be provided in the <code>changed</code> argument to give the host a hint on which part of the surface was modified with respect to the previous current surface so that the browser can choose to optimize the composition process. There is no guarantee only this rectangle will be recomposed though and the entire surface needs to be valid.


The Init and Finalize functions must be called from the main plugin thread. The SetCurrent function may be called from any thread. Reentrant calls into the plugin do not occur during a SetCurrent invocation. Therefore the browser is expected to implement SetCurrent using a locking protocol that cannot block for long. For example, SetCurrent might block only while the previous surface is being read by browser compositing.
SetCurrent can take NULL as pthe surface. This removes the previous current surface. The plugin will render nothing. This should be used before finalizing the last surface. All three functions must be called on the plugin thread.
 
Reentrant calls from the browser into the plugin do not occur during a SetCurrent invocation from the main thread. The browser is expected to implement SetCurrent using a locking protocol that cannot block for long. For example, SetCurrent might block only while the previous surface is being read by the browser's compositing.
 
= Throttling Painting =
 
To enable the plugin to match its frame rate to the browser's, the browser notifies the plugin each time it has composited a frame that used the plugin's current surface. This notification fires even if the plugin's current surface is NULL.
 
<pre>
  void NPP_DidComposite(NPP instance);
</pre>
 
== Hidden Plugins ==
 
While a plugin is in a hidden tab (or is hidden for some other reason), the browser doesn't have to fire NPP_DidComposite. When the plugin becomes visible, the browser won't wait for the plugin to re-render before compositing --- that would defeat the purpose of an asynchronous API. This could mean displaying an old, stale frame of the plugin for a moment. A plugin can mitigate this by calling SetCurrentAsyncSurface at a low rate even while DidComposite is not being received. In some situations it might be appropriate for a plugin to set the current surface to NULL while it's hidden (NPP_SetWindow having set an empty clip-rect); this would cause the plugin to render nothing instead of a stale frame when it becomes visible.


= NPDrawingModelAsyncBitmapSurface =
= NPDrawingModelAsyncBitmapSurface =


This model should be available on all platforms. It allows calls to SetCurrent on any thread.
This model should be available on all platforms.


Plugins should create a bitmap surface by calling <code>NPN_InitAsyncSurface</code> with a <code>NULL</code> value for <code>initData</code>. If an <code>NPImageFormat</code> is not supported then surface creation will fail.
Plugins should create a bitmap surface by calling <code>NPN_InitAsyncSurface</code> with a <code>NULL</code> value for <code>initData</code>. If an <code>NPImageFormat</code> is not supported then surface creation will fail.
Line 104: Line 120:
This drawing model will only be valid on Windows Vista and higher in order to simplify hardware accelerated surface sharing.
This drawing model will only be valid on Windows Vista and higher in order to simplify hardware accelerated surface sharing.


Plugins should create a Windows shared surface by calling <code>NPN_InitAsyncSurface</code> with a <code>NULL</code> value for <code>initData</code>. The resulting <code>NPAsyncSurface</code> will point to a <code>HANDLE</code> that can be used, for example, through [http://msdn.microsoft.com/en-us/library/bb173598%28v=VS.85%29.aspx OpenSharedResource] in order to create a texture for the user. In order to allow fast drawing to any hardware surfaces, the host will acquire the handle from [http://msdn.microsoft.com/en-us/library/bb174562%28v=VS.85%29.aspx IDXGISurface::GetSharedHandle]. This shared handle will represent a texture which is usable as a render target and is valid as a shader resource. The plugin can open this shared handle as a texture and then use it as a render target for any drawing operations. SetCurrent may only be called on the main thread in this model.
Plugins should create a Windows shared surface by calling <code>NPN_InitAsyncSurface</code> with a <code>NULL</code> value for <code>initData</code>. The resulting <code>NPAsyncSurface</code> will point to a <code>HANDLE</code> that can be used, for example, through [http://msdn.microsoft.com/en-us/library/bb173598%28v=VS.85%29.aspx OpenSharedResource] in order to create a texture for the user. In order to allow fast drawing to any hardware surfaces, the host will acquire the handle from [http://msdn.microsoft.com/en-us/library/bb174562%28v=VS.85%29.aspx IDXGISurface::GetSharedHandle]. This shared handle will represent a texture which is usable as a render target and is valid as a shader resource. The plugin can open this shared handle as a texture and then use it as a render target for any drawing operations. In order to synchronize GPU access to the surface the resource created will support the [http://msdn.microsoft.com/en-us/library/windows/desktop/ff471339%28v=vs.85%29.aspx IDXGIKeyedMutex] interface, the key used for both acquiring and releasing sync is 0.
 
Plugins should finalize Windows shared surfaces by calling <code>NPN_FinalizeAsyncSurface</code> when they are done with the surface.
 
== Creating the Device ==


When a surface is set as current the plugin is responsible for making sure all drawing calls on that surface have completed execution!
On dual-GPU systems, the browser and plugin process may unintentionally bind to different GPUs if default DXGI parameters are used. When this happens the plugin's draw calls will most likely fail, or crash. To address this, it is necessary to open the browser's shared DXGI surfaces on the same adapter. The adapter can be found using <tt>NPNVpreferredDXGIAdapter</tt>. For example,


Plugins should finalize Windows shared surfaces by calling <code>NPN_FinalizeAsyncSurface</code> when they are done with the surface.
<pre>
static IDXGIAdapter1*
FindDXGIAdapter(NPP npp, IDXGIFactory1* factory)
{
  DXGI_ADAPTER_DESC preferred;
  if (NPN_GetValue(npp, NPNVpreferredDXGIAdapter, &preferred) != NPERR_NO_ERROR) {
    return NULL;
  }
 
  for (UINT index = 0; ; index++) {
    IDXGIAdapter* adapter = NULL;
    if (FAILED(factory1->EnumAdapters1(index, &adapter)) || !adapter) {
      return nullptr;
    }
 
    DXGI_ADAPTER_DESC desc;
    if (SUCCEEDED(adapter->GetDesc(&desc)) &&
        desc.AdapterLuid.LowPart == preferred.AdapterLuid.LowPart &&
        desc.AdapterLuid.HighPart == preferred.AdapterLuid.HighPart)
    {
      return adapter;
    }
    adapter->Release();
  }
  return NULL;
}
</pre>
 
The adapter returned by this function can then be used with any DXGI device creation function, such as <tt>D3D10CreateDevice</tt>. Failure to negotiate the adapter properly (or failure to use an IDXGIAdapter1) will result in undefined behavior.
 
If no matching adapter can be found, the safest course of action for the plugin is to switch to a different drawing model (such the AsyncBitmap model).
 
== Sample Code ==


This sample code illustrates usage of this drawing model:
This sample code illustrates usage of this drawing model:


<pre>
  ID3D10Device *pDevice10;
  ID3D10Device *pDevice10;
// Initialize device.
  NPAsyncSurface *npFrontBuffer = new NPAsyncSurface;
  NPAsyncSurface *npFrontBuffer = new NPAsyncSurface;
  NPAsyncSurface *npBackBuffer = new NPAsyncSurface;
  NPAsyncSurface *npBackBuffer = new NPAsyncSurface;
Line 120: Line 171:
  ID3D10Texture2D *backBuffer;
  ID3D10Texture2D *backBuffer;
   
   
  NPSize size;
  void Init() {
size.width = pluginwidth;
  ... Initialize pDevice10 ...
size.height = pluginheight;
   
   
NPN_InitAsyncSurface(instance, size, NPImageFormatBGRX32, NULL, npFrontBuffer);
  NPSize size;
NPN_InitAsyncSurface(instance, size, NPImageFormatBGRX32, NULL, npBackBuffer);
  size.width = pluginwidth;
  size.height = pluginheight;
   
   
pDevice10->OpenSharedResource(npFrontBuffer->handle,
  NPN_InitAsyncSurface(instance, size, NPImageFormatBGRX32, NULL, npFrontBuffer);
                              __uuidof(ID3D10Texture2D),
  NPN_InitAsyncSurface(instance, size, NPImageFormatBGRX32, NULL, npBackBuffer);
                              (void**)(&frontBuffer));
pDevice10->OpenSharedResource(npBackBuffer->handle,
                              __uuidof(ID3D10Texture2D),
                              (void**)(&backBuffer));
   
   
while (painting) {
   pDevice10->OpenSharedResource(npFrontBuffer->handle,
  // Draw to backBuffer texture
                                __uuidof(ID3D10Texture2D),
   pDevice10->Flush();
                                (void**)(&frontBuffer));
  NPN_SetCurrentAsyncSurface(instance, npBackBuffer);
   pDevice10->OpenSharedResource(npBackBuffer->handle,
  ID3D10Texture2D *tmp = frontBuffer;
                                __uuidof(ID3D10Texture2D),
  NPAsyncSurface *npTmp = npFrontBuffer;
                                (void**)(&backBuffer));
  frontBuffer = backBuffer;
   DrawFrame();
   npFrontBuffer = npBackBuffer;
  backBuffer = tmp;
   npBackBuffer = npTmp;
  }
  }
   
   
  frontBuffer->Release();
  void DrawFrame() {
backBuffer->Release();
  IDXGIKeyedMutex *mutex;
  backBuffer->QueryInterface(&mutex);
  mutex->AcquireSync(0);
   
   
NPN_SetCurrentAsyncSurface(instance, NULL);
  ... Draw to backBuffer texture ...
NPN_FinalizeAsyncSurface(instance, npFrontBuffer);
NPN_FinalizeAsyncSurface(instance, npBackBuffer);
   
   
delete npFrontBuffer;
  mutex->ReleaseSync(0);
delete npBackBuffer;
  mutex->Release();
 
= NPDrawingModelAsyncWindowsDX9ExSurface =
 
This drawing model will only be valid on Windows Vista and higher in order to simplify hardware accelerated surface sharing for plugins wanting to use Direct3D 9Ex.
 
In this drawing model the plugin will be responsible for actual creation of the texture. The reason for this behavior is that textures created in Direct3D 10 cannot be shared using Direct3D 9/10 interoperability. Textures created with Direct3D 9 can be opened in Direct3D 10. In this model the plugin will create a texture using the <code>CreateTexture</code> call on a Direct3D 9Ex device. The <code>pSharedHandle</code> argument will be used to acquire a shared handle for that texture. This shared handle should be set as the <code>data</code> member of the <code>NPAsyncSurface</code> structure before <code>NPN_InitAsyncSurface</code> is called to initialize the surface. The plugin must also set the <code>stride</code> member of <code>NPAsyncSurface</code> before calling <code>NPN_InitAsyncSurface</code>.
 
The return value of the <code>NPN_InitAsyncSurface</code> call will indicate if the surface was successfully accessed by the host. When a surface is set current the plugin must make sure all drawing calls to that surface have finished execution. For Direct3D 9Ex this means making sure that the GPU has processed all graphics command queued to draw to that surface in order to prevent race condition with a host using another device. One way of doing this is do use a <code>D3DQUERYTYPE_EVENT</code> query to determine when the graphics pipeline has finished executing all events. Another method is to execute a blocking readback from a small (i.e. 1x1 pixel) portion of the texture. When the readback has successfully executed then the GPU has finished the drawing calls to the surface. Failure to follow these guidelines may result in incomplete drawing of the contents of the texture. SetCurrent may only be called on the main thread in this model.


Plugins should finalize surfaces by calling <code>NPN_FinalizeAsyncSurface</code>.
IDirect3DDevice9Ex *pDevice9;
// Initialize device.
NPAsyncSurface *npFrontBuffer = new NPAsyncSurface;
NPAsyncSurface *npBackBuffer = new NPAsyncSurface;
IDirect3DTexture9 *frontBuffer;
IDirect3DTexture9 *backBuffer;
NPSize size;
size.width = pluginwidth;
size.height = pluginheight;
HANDLE frontBufferHandle = NULL;
HANDLE backBufferHandle = NULL;
pDevice9->CreateTexture(pluginWidth, pluginHeight, 0,
                        D3DUSAGE_RENDERTARGET,
                        D3DFMT_X8R8G8B8, D3DPOOL_DEFAULT,
                        &frontBuffer, &frontBufferHandle);
pDevice9->CreateTexture(pluginWidth, pluginHeight, 0,
                        D3DUSAGE_RENDERTARGET,
                        D3DFMT_X8R8G8B8, D3DPOOL_DEFAULT,
                        &backBuffer, &backBufferHandle);
NPN_InitAsyncSurface(instance, size, NPImageFormatBGRX32, frontBufferHandle, npFrontBuffer);
NPN_InitAsyncSurface(instance, size, NPImageFormatBGRX32, backBufferHandle, npBackBuffer);
while (painting) {
  // Draw to backBuffer texture
  // Synchronize by one of the methods described above
   NPN_SetCurrentAsyncSurface(instance, npBackBuffer);
   NPN_SetCurrentAsyncSurface(instance, npBackBuffer);
   IDirect3DDevice9 *tmp = frontBuffer;
   ID3D10Texture2D *tmp = frontBuffer;
   NPAsyncSurface *npTmp = npFrontBuffer;
   NPAsyncSurface *npTmp = npFrontBuffer;
   frontBuffer = backBuffer;
   frontBuffer = backBuffer;
Line 206: Line 209:
  }
  }
   
   
  frontBuffer->Release();
  void NPP_DidComposite() {
backBuffer->Release();
  DrawFrame();
}
void Shutdown() {
  frontBuffer->Release();
  backBuffer->Release();
   
   
NPN_SetCurrentAsyncSurface(instance, NULL);
  NPN_SetCurrentAsyncSurface(instance, NULL);
NPN_FinalizeAsyncSurface(instance, npFrontBuffer);
  NPN_FinalizeAsyncSurface(instance, npFrontBuffer);
NPN_FinalizeAsyncSurface(instance, npBackBuffer);
  NPN_FinalizeAsyncSurface(instance, npBackBuffer);
   
   
delete npFrontBuffer;
  delete npFrontBuffer;
delete npBackBuffer;
  delete npBackBuffer;
 
}
= Open Issues =
</pre>
 
* Add accelerated Linux drawing model.
* How would the last surface be made non-current in order to be deleted? NPN_FinalizeAsyncSurface says that it will return an error if the surface is current, but there doesn't appear to be any way to set no surface as current.
Dvander
Confirmed users
156

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/373613...1105676"