NPAPI:DrawImage: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
No edit summary
 
(23 intermediate revisions by 2 users not shown)
Line 1: Line 1:
= Problem Summary =
= Status =


It is very painful to render plugin content to non-X surfaces on Linux using X11. (For Mozilla if MOZ_X11 is defined then HandleEvent is operating with XEvent's).
Under consideration.
When browser renders its content into a memory buffer or to some other surface (Qt::RasterPaintEngine/cairo-image-surface which are not very friendly to X), then we are forced to do data moving X-client-X...


= Proposal =
= Contributors =


I propose an NPAPI Drawing model which would allow a plugin to render directly into an image memory buffer, without involving XSurface.
* Last modified: May 12, 2010
* Authors: Oleg Romashin (Nokia), Josh Aas (Mozilla)
* Contributors: Robert O'Callahan (Mozilla)


= Event model negotiation =
= Overview =


For documentation on negotiating event models, see [[NPAPI:Models]]. The event model variables for all platforms are:
This drawing model allows plugins to draw into a buffer.


<pre>
= Event Model Requirements =
* NPEventModelNative - (Native platform default events, X/HWND... )
* NPEventModelIndependent
* NPNVsupportsNativeBool (NPNVariable = 3000)
* NPNVsupportsIndependentBool (NPNVariable = 3001)
</pre>
 
We should provide the possibility to implement a different event model for all platforms (not only Mac OS X).
 
== NPEventModelNative ==
 
Plugin and browser consider event structure as native event pointer, and Handling works as it is now by default in windowless mode.
 
== NPEventModelIndependent ==
 
<pre>
typedef struct _NPEvent
{
  void*        event;      /* event structure pointer */
  NPEventType  type;        /* typ of event, defined in NPEventType */
} NPEvent;
</pre>
 
<pre>
typedef enum {
  NPEventType_Native = 0x0001,  /* Native event structure specific to platform (XEvent, HWND...)
  NPEventType_DrawImage = 0x0002,    /* Draw Image even type, event structure == NPImageData.
  NPEventType_GetFocusEvent = ....
  ............
  Other events which are need to be platform independent
  ............
} NPEventType;
</pre>
 
== NPEventModelType negotiation ==
 
Browser and plugin should negotiate Event model type:
* if no supported model type defined or NPEventModelNative only, then we use the default event model system, specific to platform
* If NPEventModelIndependent was negotiated as supported, then browser and plugin should also negotiate the list of supported events


=== NPEventType negotiation ===
This drawing model is currently compatible with the following event models:


<pre>
* [[NPAPI:ExtendedXEventModel|Extended X Event Model]]
NPNVsupportsEventTypes = 4000


NPEventType aTypes;
While this is currently only compatible with the extended X event model, this could change. This drawing model was designed to work on any platform.
browser->getvalue(instance, NPNVsupportsEventTypes, &aTypes)
</pre>


Then plugin should decide which event supported by plugin and browser (remove bits which are not supported by plugin), and set value to browser:
= NPDrawingModelX =


<pre>
As this will be the first alternative to the default X drawing model, we'll designate the original model:
browser->setvalue(instance, NPNVsupportsEventTypes, commonTypes);
</pre>


= Drawing Model =
* NPDrawingModelX (NPDrawingModel = 4)
* NPNVsupportsXDrawingBool (NPNVariable = 2004)


For documentation on negotiating drawing model, see NPAPI:Models. The drawing model variables for Platform independent image rendering are:
= NPDrawingModelDrawImage =


* NPDrawingModelImage (NPDrawingModel = 0)
For documentation on negotiating drawing models, see [[NPAPI:Models]]. The drawing model variables for draw image events are:
* NPNVsupportsDrawImageBool (NPNVariable = 2000)


== NPDrawingModelImage ==
* NPDrawingModelDrawImage (NPDrawingModel = 5)
 
* NPNVsupportsDrawImageBool (NPNVariable = 3005)
Using NPImageData structure as event
where NPImageData:
<pre>
typedef struct _NPImageData
{
  /* Image data parameters */
  char*        data;      /* image pointer */
  int32_t      stride;    /* Stride of data image pointer */
  NPImageFormat format;    /* Format of image pointer */
  NPSize        dataSize;  /* Data buffer size */
  /* Clip rectangle, must be used for trusted plugins */
  int32_t      x;          /* Expose x relative to 0,0 of plugin window area*/
  int32_t      y;          /* Expose y relative to 0,0 of plugin window area */
  uint32_t      width;      /* Expose width */
  uint32_t      height;    /* Expose height */
  /* Position and scale values for plugin area */
  float        translateX; /* translate X matrix value, (x offset) */
  float        translateY; /* translate Y matrix value, (y offset) */
  /* Defines plugin window size on scaled context, if 1 then size = window size */
  float        scaleX;    /* scale X matrix value */
  float        scaleY;    /* scale Y matrix value */
} NPImageData;
</pre>


where NPImageFormat (similar to the current Cairo image format list):
= Event Structure =
<pre>
typedef enum {
  NPImageFormatARGB32 = 0x0001,
  NPImageFormatRGB24 = 0x0002,
  NPImageFormatRGB16_565 = 0x0004,
  NPImageFormatA8 = 0x0008,
  NPImageFormatA1 = 0x0010
} NPImageFormat;
</pre>


=== NPImageFormat type negotiation ===
  typedef enum {
    /* each pixel is a 32-bit quantity, with
    * alpha in the upper 8 bits, then red, then green, then blue.
    * The 32-bit quantities are stored native-endian. Pre-multiplied
    * alpha is used. (That is, 50% transparent red is 0x80800000,
    * not 0x80ff0000.) */
    NPImageFormatARGB32_pre = 0x1,
    /* each pixel is a 32-bit quantity, with
    * the upper 8 bits unused. Red, Green, and Blue are stored
    * in the remaining 24 bits in that order. */
    NPImageFormatRGB24      = 0x2,  /* x8r8g8b8 */
    NPImageFormatRGB16_565  = 0x4  /* r5g6b5  */
    /* non-premultiplied version of RGBA32 format */
    NPImageFormatARGB32    = 0x8,
    /* can be extended */
  } NPImageFormat;
 
  typedef struct _NPImageData
  {
    /* Image data parameters */
    char*        data;      /* image pointer */
    int32_t      stride;    /* Stride of data image pointer */
    NPImageFormat format;    /* Format of image pointer */
    /* size of plugin window in current buffer, used for scale information */
    NPSize        pluginSize;
    /* "data" points to exposeX/Y point, which is also offset in plugin window area */
    int32_t      exposeX;
    int32_t      exposeY;
    /* size of data buffer, size of area where plugin should paint */
    /* exposeX/Y + exposeSize <= pluginSize */
    NPSize        exposeSize;
  } NPImageData;


How to initialize cairo destination surface with this info:
<pre>
<pre>
NPNVsupportsImageFormat = 5000,
cairo_surface_t *src = plugin_window_image (probably cached)
</pre>
cairo_surface_t *dest =
  cairo_image_surface_create_for_data((unsigned char*)event->data,
                                      CAIRO_FORMAT_XXX(event->format),
                                      event->exposeSize.width,
                                      event->exposeSize.height,
                                      event->stride);


Implementation is similar to Event types negotiation
cairo_t *cr = cairo_create(dest);
 
cairo_set_source_surface (cr, src, -event->exposeRect.left, -event->exposeRect.top);
<pre>
NPImageFormat aFormats;
browser->getvalue(instance, NPNVsupportsImageFormat, &aFormats)
</pre>
</pre>


Then the plugin should decide which formats supported by plugin and browser (remove bits which are not supported by plugin), and send a value to browser:
= NPImageFormat Type Negotiation =


<pre>
* NPNVsupportedImageFormats (NPNVariable = 19)
browser->setvalue(instance, NPNVsupportsImageFormat, commonFormats);
* NPPVImageFormats (NPPVariable = 22)
</pre>


And then browser should use only formats enumerated in commonFormats value.
Negotiating the image format will work similarly to [[NPAPI:Models|drawing and event model negotiation]], but the value of NPNVsupportedImageFormats will be a bitmap of supported types. Based on the browser's supported formats, the plugin should select a subset which it can handle. The format for any given event will be specified in the event, and will be from the plugin's supported subset.


According to plugin state (transparency), and platform preffered depth/image-format browser will send event to plugin with specific format.
  NPImageFormat supportedFormats;
  browserFuncs->getvalue(instance, NPNVsupportedImageFormats, &supportedFormats);
  ...
  browserFuncs->setvalue(instance, NPPVImageFormats, (void*)supportedSubset);

Latest revision as of 14:48, 12 May 2010

Status

Under consideration.

Contributors

  • Last modified: May 12, 2010
  • Authors: Oleg Romashin (Nokia), Josh Aas (Mozilla)
  • Contributors: Robert O'Callahan (Mozilla)

Overview

This drawing model allows plugins to draw into a buffer.

Event Model Requirements

This drawing model is currently compatible with the following event models:

While this is currently only compatible with the extended X event model, this could change. This drawing model was designed to work on any platform.

NPDrawingModelX

As this will be the first alternative to the default X drawing model, we'll designate the original model:

  • NPDrawingModelX (NPDrawingModel = 4)
  • NPNVsupportsXDrawingBool (NPNVariable = 2004)

NPDrawingModelDrawImage

For documentation on negotiating drawing models, see NPAPI:Models. The drawing model variables for draw image events are:

  • NPDrawingModelDrawImage (NPDrawingModel = 5)
  • NPNVsupportsDrawImageBool (NPNVariable = 3005)

Event Structure

 typedef enum {
   /* each pixel is a 32-bit quantity, with
    * alpha in the upper 8 bits, then red, then green, then blue.
    * The 32-bit quantities are stored native-endian. Pre-multiplied
    * alpha is used. (That is, 50% transparent red is 0x80800000,
    * not 0x80ff0000.) */
   NPImageFormatARGB32_pre = 0x1,
   /* each pixel is a 32-bit quantity, with
    * the upper 8 bits unused. Red, Green, and Blue are stored
    * in the remaining 24 bits in that order. */
   NPImageFormatRGB24      = 0x2,  /* x8r8g8b8 */
   NPImageFormatRGB16_565  = 0x4   /* r5g6b5   */
   /* non-premultiplied version of RGBA32 format */
   NPImageFormatARGB32     = 0x8,
   /* can be extended */
 } NPImageFormat;
 
 typedef struct _NPImageData
 {
   /* Image data parameters */
   char*         data;       /* image pointer */
   int32_t       stride;     /* Stride of data image pointer */
   NPImageFormat format;     /* Format of image pointer */
   /* size of plugin window in current buffer, used for scale information */
   NPSize        pluginSize; 
   /* "data" points to exposeX/Y point, which is also offset in plugin window area */
   int32_t       exposeX; 
   int32_t       exposeY; 
   /* size of data buffer, size of area where plugin should paint */
   /* exposeX/Y + exposeSize <= pluginSize */
   NPSize        exposeSize; 
 } NPImageData;

How to initialize cairo destination surface with this info:

cairo_surface_t *src = plugin_window_image (probably cached)
cairo_surface_t *dest =
  cairo_image_surface_create_for_data((unsigned char*)event->data,
                                      CAIRO_FORMAT_XXX(event->format),
                                      event->exposeSize.width,
                                      event->exposeSize.height,
                                      event->stride);

cairo_t *cr = cairo_create(dest);
cairo_set_source_surface (cr, src, -event->exposeRect.left, -event->exposeRect.top);

NPImageFormat Type Negotiation

  • NPNVsupportedImageFormats (NPNVariable = 19)
  • NPPVImageFormats (NPPVariable = 22)

Negotiating the image format will work similarly to drawing and event model negotiation, but the value of NPNVsupportedImageFormats will be a bitmap of supported types. Based on the browser's supported formats, the plugin should select a subset which it can handle. The format for any given event will be specified in the event, and will be from the plugin's supported subset.

 NPImageFormat supportedFormats;
 browserFuncs->getvalue(instance, NPNVsupportedImageFormats, &supportedFormats);
 ...
 browserFuncs->setvalue(instance, NPPVImageFormats, (void*)supportedSubset);