Platform/GFX/Gralloc: Difference between revisions

Jump to navigation Jump to search

Platform/GFX/Gralloc (view source)

Revision as of 19:33, 22 August 2013

2,757 bytes added ,  22 August 2013
→‎Genlock
 
(15 intermediate revisions by 3 users not shown)
Line 44: Line 44:


Content side:
Content side:
* Entry point: PLayersTransactionChild::SendPGrallocBufferConstructor (generally called by ISurfaceAllocator::AllocGrallocBuffer).
* Entry point: PLayerTransactionChild::SendPGrallocBufferConstructor (generally called by ISurfaceAllocator::AllocGrallocBuffer).
* This sends a synchronous IPC message to the compositor side.
* This sends a synchronous IPC message to the compositor side.




Over to the compositor side:
Over to the compositor side:
* The message is received and this comes in as a call to PLayerTransactionParent::AllocPGrallocBuffer, implemented in LayersTransactionParent.cpp.
* The message is received and this comes in as a call to PLayerTransactionParent::AllocPGrallocBuffer, implemented in LayerTransactionParent.cpp.
* This calls GrallocBufferActor::Create(...), which actually creates the GraphicBuffer* and a GrallocBufferActor* (The GrallocBufferActor contains a sp<GraphicBuffer> that references the newly-created GraphicBuffer*).
* This calls GrallocBufferActor::Create(...), which actually creates the GraphicBuffer* and a GrallocBufferActor* (The GrallocBufferActor contains a sp<GraphicBuffer> that references the newly-created GraphicBuffer*).
* GrallocBufferActor::Create returns the GrallocBufferActor as a PGrallocBufferParent*, and the GraphicBuffer* as a MaybeMagicGrallocBufferHandle.
* GrallocBufferActor::Create returns the GrallocBufferActor as a PGrallocBufferParent*, and the GraphicBuffer* as a MaybeMagicGrallocBufferHandle.
Line 69: Line 69:
Most of our gralloc buffers are owned in this way by GrallocBufferActor's. The question then becomes, what controls the lifetime of GrallocBufferActors?
Most of our gralloc buffers are owned in this way by GrallocBufferActor's. The question then becomes, what controls the lifetime of GrallocBufferActors?


GrallocBufferActors are "managed" by IPDL-generated code. When they are created by the above-described protocol, as said above, they are added to the "managee lists" of the LayerTransactionParent on the compositor side, and of the LayerTransactionParent on the content side.
GrallocBufferActors are "managed" by IPDL-generated code. When they are created by the above-described protocol, as said above, they are added to the "managee lists" of the LayerTransactionParent on the compositor side, and of the LayerTransactionChild on the content side.


GrallocBufferActors are destroyed when either a "delete" IPC message is sent (see: Send__delete__) or the top-level IPDL manager goes away.
GrallocBufferActors are destroyed when either a "delete" IPC message is sent (see: Send__delete__) or the top-level IPDL manager goes away.
Line 86: Line 86:


How gralloc buffer locking works, varies greatly between drivers. While we only directly deal with the gralloc API, which is the same on all Android devices (android::GraphicBuffer::lock and unlock), the precise lock semantics vary between different vendor-specific lock mechanisms, so we need to pay specific attention to them.
How gralloc buffer locking works, varies greatly between drivers. While we only directly deal with the gralloc API, which is the same on all Android devices (android::GraphicBuffer::lock and unlock), the precise lock semantics vary between different vendor-specific lock mechanisms, so we need to pay specific attention to them.
* On Android >= 4.2, a standardized fence mechanism is used, that should work uniformly across all drivers. We do not yet support it. B2G does not yet use Android 4.2.
* On Android >= 4.2, a standardized fence mechanism is used, that should work uniformly across all drivers. We do not yet support it. B2G does not yet use Android 4.2. These are called sync points and are discussed here [http://source.android.com/devices/graphics.html] and [https://android.googlesource.com/kernel/common/+/android-3.4/Documentation/sync.txt]. They are currently in the staging tree [https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/staging/android/sync.c] and there is a similar non-android linux concept called dma-buf fences being worked on.
* On Qualcomm hardware pre-Android-4.2, a Qualcomm-specific mechanism, named Genlock, is used. We explicitly support it. More on this below.
* On Qualcomm hardware pre-Android-4.2, a Qualcomm-specific mechanism, named Genlock, is used. We explicitly support it. More on this below.
* On non-Qualcomm, pre-Android-4.2 hardware, other vendor-specific mechanisms are used, which we do not support (see e.g. {{bug|871624}}).
* On non-Qualcomm, pre-Android-4.2 hardware, other vendor-specific mechanisms are used, which we do not support (see e.g. {{bug|871624}}).
Line 104: Line 104:


Genlock is implemented in the kernel. The kernel GL driver is able to lock and unlock directly. Typically, it will place a read lock on any gralloc buffer that's bound to a texture it's sampling from, and unlock when it's done with that texture.
Genlock is implemented in the kernel. The kernel GL driver is able to lock and unlock directly. Typically, it will place a read lock on any gralloc buffer that's bound to a texture it's sampling from, and unlock when it's done with that texture.
A logging patch for genlock that lets you see locking across process is here: [http://people.mozilla.com/~jmuizelaar/genlock-logging-patch]


== How we lock/unlock Gralloc buffers ==
== How we lock/unlock Gralloc buffers ==
Line 137: Line 139:
= How Android is using Gralloc =
= How Android is using Gralloc =


== EGLSurface, ANativeWindow, etc. ==
== Some terminology: EGLSurface, ANativeWindow, etc. ==


'''EGLSurface''' is a portable EGL abstraction for a possibly multi-buffered render target.
'''EGLSurface''' is a portable EGL abstraction for a possibly multi-buffered render target.
Line 171: Line 173:
== SurfaceTexture ==
== SurfaceTexture ==


SurfaceTexture is the server side of a client-server system, whose client side is SurfaceTextureClient. As explained above, SurfaceTextureClient is a concrete implementation of ANativeWindow.
'''SurfaceTexture''' is the server side of a client-server system, whose client side is '''SurfaceTextureClient'''. As explained above, SurfaceTextureClient is a concrete implementation of ANativeWindow.


The reason to use a client-server system like this is to allow producing and compositing a surface in two different processes.
The reason to use a client-server system like this is to allow producing and compositing a surface in two different processes.


Since eglSwapBuffers calls to ANativeWindow::dequeueBuffer to acquire new buffer and ANativeWindow::queueBuffer to return the buffer implicitly, the user can avoid to create several textures and manage them directly. The other useful feature of implement it in this way is that most BSP vendor provided graphics profilers(e.g. Adreno profiler from QCOM, PerfHUD ES from nVidia) that recognize the eglSwapBuffers call as frame boundary to collect frame based GL information from driver to help developement and performance tuning. SurfaceTexture is a unified buffer management mechanism on Android, which you can setup it to run in different mode: sync/async, single/multiple buffer by simple function call which make SurfaceTexture be used everywhere in Android: Codec/Camera configure it to have serveral buffer (based on hardware, 9 for camera preview in Unagi) run asynchronously, Layer rendering configure it to have 2~3 buffer (based on BSP) run synchronously. And since its flexbility, you can encode a OpenGL rendered surface on-the-fly by using SurfaceMediaSource (which implement ISurfaceTexture) which is the core of Google Miracast.
Let us introduce two important functions that a client needs to call on its ANativeWindow: dequeueBuffer and queueBuffer
* '''dequeueBuffer''' acquires a new buffer for the client to draw to;
* '''queueBuffer''' lets the client indicate that it has finished drawing to a buffer, and queues it (e.g. for compositing).
 
 
Since eglSwapBuffers internally calls dequeueBuffer and queueBuffer, this system removes the need for manual management of GraphicBuffer's as we are currently doing in our B2G code.
 
The other benefit of this system is that most [http://en.wikipedia.org/wiki/Board_support_package BSP] vendors provide graphics profilers (e.g. Adreno Profiler from QCOM, PerfHUD ES from nVidia) which recognize the eglSwapBuffers calls as frame boundaries to collect frame-based GL information from the driver to help development and performance tuning.
 
In Android 2, there were many buffer management systems. In Android 4, all of this is unified under SurfaceTexture. This is made possible by the great flexibility of SurfaceTexture:
* SurfaceTexture supports both synchronous and asynchronous modes.
* SurfaceTexture supports generic multi-buffering: it can have any number of buffers between 1 and 32.
 
 
Examples:
* The Codec/Camera code configures it to have several buffers (depending on hardware, for instance 9 buffers for camera preview on Unagi) and run asynchronously
* SurfaceFlinger (the Android compositor) configures it to have 2--3 buffers (depending on [http://en.wikipedia.org/wiki/Board_support_package BSP]) and run synchronously.
* Google Miracast uses it to encode OpenGL-rendered surfaces on-the-fly.
 
 
Let us now describe how the client-server SurfaceTexture system allocates GraphicBuffer's, and how both the client and server sides keep track of these shared buffer handles. Again, SurfaceTexture is the server-side class, while SurfaceTextureClient is the client-side class. Each of them stores an array of GraphicBuffers, which is called mSlots in both classes. The GraphicBuffer objects are separate instances in SurfaceTexture and in SurfaceTextureClient, but the underlying buffer handles are the same. The mechanism here is as follows. The client side issues a SurfaceTextureClient::dequeueBuffer call to get a new buffer to paint to. If there is not already a free buffer in mSlots, and the number of buffers is under the limit (e.g. 3 for triple buffering), it sends an IPC message that results in a call to SurfaceTexture::dequeueBuffer which allocates a GraphicBuffer. After this transaction, still inside of SurfaceTextureClient::dequeueBuffer, another IPC message is sent, that results in a call to SurfaceTexture::requestBuffer to get the GraphicBuffer serialized over IPC back to it, using GraphicBuffer::flatten and GraphicBuffer::unflatten, and cache it into its own mSlots. The mSlots arrays on both sides mirror each other, so that the two sides can refer to GraphicBuffer's by index. This allows the client and server side to communicate with each other by passing only indices, without flattening/unflattening GraphicBuffers again and again.


SurfaceTexture is the server-side class. SurfaceTextureClient is the client-side class. Both store an array of GraphicBuffers, called mSlots. The GraphicBuffer objects are separate instances in SurfaceTexture and in SurfaceTextureClient, but the underlying gralloc buffer handles are the same. The mechanism here is the client side issues a SurfaceTextureClient::dequeueBuffer call to get a new buffer to paint to. If there are not already enough buffers, and the number of buffers is not over the limit (32), it sends an IPC message that results in a call to SurfaceTexture::dequeueBuffer which allocates the GraphicBuffer. And after buffer allocation, the client side will send an IPC message that results in a call to SurfaceTexture::requestBuffer to get the gralloc buffer handle serialized over IPC back to it, and construct a GraphicBuffer around this handle, and cache it into its own mSlots. The mSlots arrays on both sides mirror each other, so that the two sides can refer to GraphicBuffers by index.
Let us now describe what happens in SurfaceTextureClient::dequeueBuffer when there is no free buffer and the number of buffers has already met the limit (e.g. 3 for triple buffering). In this case, the server side (SurfaceTexture::dequeueBuffer) will wait for a buffer to be queued, and the client side waits for that, so that SurfaceTextureClient::dequeueBuffer will not return until a buffer has actually been queued on the server side. This is what allows SurfaceTexture to support both synchronous and asynchronous modes with the same API.


When the client side calls ANativeWindow::queue to present the frame from eglSwapBuffers call, SurfaceTextureClient::queue is issued and send the index where the rendered buffer is to server side SurfaceTexture::queue. Which cause the index queued into SurfaceTexture::mQueue for rendering. SurfaceTexture::mQueue is a wait queue for frame that wants to be rendered. In sync mode, the frame are showed one after another, but may dropped in async mode.
Let us now explain how synchronous mode works. In synchronous mode, on the client side, inside of eglSwapBuffers, when ANativeWindow::queueBuffer is called to present the frame, it sends the index of the rendered buffer to the server side. This causes the index to be queued into SurfaceTexture::mQueue for rendering. SurfaceTexture::mQueue is a wait queue for frames that want to be rendered. In synchronous mode, all the frames are shown one after another, whereas in asynchronous mode frames may be dropped.


For SurfaceFlinger, after each time SurfaceTexture::queue are issued, it will start for next renderloop. In each render loop, SurfaceFlinger calls to SurfaceTexture::updateTexImage to dequeue a frame from SurfaceTexture::mQueue and bind the GraphicBuffer into a texture.
Let us now explain how SurfaceFlinger (the Android compositor) uses this system to get images onto the screen. Each time SurfaceTexture::queueBuffer is called, it causes SurfaceFlinger to start the next iteration of the render loop. In each iteration, SurfaceFlinger calls SurfaceTexture::updateTexImage to dequeue a frame from SurfaceTexture::mQueue and bind that GraphicBuffer into a texture, just like we do in GrallocTextureHostOGL::Lock.


As a result, Android does not explicit unlock the GraphicBuffer as B2G did now. Since it will lock next buffer each time new buffer comes. And it will use eglCreateSyncKHR/eglClientWaitSyncKHR to make sure the buffer are not locked before it returns the GraphicBuffer into the buffer pool.
The magic part is that SurfaceFlinger does not need to do the equivalent of GrallocTextureHostOGL::Unlock. In our case, we have a separate OpenGL texture object for each TextureHost, which typically (at least in the case of a ContentHost) represent one buffer each (so a double-buffered ContentHost has two TextureHost's). So we have to unbind the GraphicBuffer from the OpenGL texture before we can hand it back to the content side --- otherwise it would remain locked for read and couldn't be locked for write for content drawing. By contrast, SurfaceFlinger does not need to worry about this because it uses only one OpenGL texture, so that when it binds a new GraphicBuffer to it for compositing, that automatically unbinds the previous one!
Jrmuizel
Confirmed users
523

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/659029...698336"

Navigation menu