Confirmed users
523
edits
Platform/GFX/Gralloc: Difference between revisions
→Genlock
| (15 intermediate revisions by 3 users not shown) | |||
| Line 44: | Line 44: | ||
Content side: | Content side: | ||
* Entry point: | * Entry point: PLayerTransactionChild::SendPGrallocBufferConstructor (generally called by ISurfaceAllocator::AllocGrallocBuffer). | ||
* This sends a synchronous IPC message to the compositor side. | * This sends a synchronous IPC message to the compositor side. | ||
Over to the compositor side: | Over to the compositor side: | ||
* The message is received and this comes in as a call to PLayerTransactionParent::AllocPGrallocBuffer, implemented in | * The message is received and this comes in as a call to PLayerTransactionParent::AllocPGrallocBuffer, implemented in LayerTransactionParent.cpp. | ||
* This calls GrallocBufferActor::Create(...), which actually creates the GraphicBuffer* and a GrallocBufferActor* (The GrallocBufferActor contains a sp<GraphicBuffer> that references the newly-created GraphicBuffer*). | * This calls GrallocBufferActor::Create(...), which actually creates the GraphicBuffer* and a GrallocBufferActor* (The GrallocBufferActor contains a sp<GraphicBuffer> that references the newly-created GraphicBuffer*). | ||
* GrallocBufferActor::Create returns the GrallocBufferActor as a PGrallocBufferParent*, and the GraphicBuffer* as a MaybeMagicGrallocBufferHandle. | * GrallocBufferActor::Create returns the GrallocBufferActor as a PGrallocBufferParent*, and the GraphicBuffer* as a MaybeMagicGrallocBufferHandle. | ||
| Line 69: | Line 69: | ||
Most of our gralloc buffers are owned in this way by GrallocBufferActor's. The question then becomes, what controls the lifetime of GrallocBufferActors? | Most of our gralloc buffers are owned in this way by GrallocBufferActor's. The question then becomes, what controls the lifetime of GrallocBufferActors? | ||
GrallocBufferActors are "managed" by IPDL-generated code. When they are created by the above-described protocol, as said above, they are added to the "managee lists" of the LayerTransactionParent on the compositor side, and of the | GrallocBufferActors are "managed" by IPDL-generated code. When they are created by the above-described protocol, as said above, they are added to the "managee lists" of the LayerTransactionParent on the compositor side, and of the LayerTransactionChild on the content side. | ||
GrallocBufferActors are destroyed when either a "delete" IPC message is sent (see: Send__delete__) or the top-level IPDL manager goes away. | GrallocBufferActors are destroyed when either a "delete" IPC message is sent (see: Send__delete__) or the top-level IPDL manager goes away. | ||
| Line 86: | Line 86: | ||
How gralloc buffer locking works, varies greatly between drivers. While we only directly deal with the gralloc API, which is the same on all Android devices (android::GraphicBuffer::lock and unlock), the precise lock semantics vary between different vendor-specific lock mechanisms, so we need to pay specific attention to them. | How gralloc buffer locking works, varies greatly between drivers. While we only directly deal with the gralloc API, which is the same on all Android devices (android::GraphicBuffer::lock and unlock), the precise lock semantics vary between different vendor-specific lock mechanisms, so we need to pay specific attention to them. | ||
* On Android >= 4.2, a standardized fence mechanism is used, that should work uniformly across all drivers. We do not yet support it. B2G does not yet use Android 4.2. | * On Android >= 4.2, a standardized fence mechanism is used, that should work uniformly across all drivers. We do not yet support it. B2G does not yet use Android 4.2. These are called sync points and are discussed here [http://source.android.com/devices/graphics.html] and [https://android.googlesource.com/kernel/common/+/android-3.4/Documentation/sync.txt]. They are currently in the staging tree [https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/staging/android/sync.c] and there is a similar non-android linux concept called dma-buf fences being worked on. | ||
* On Qualcomm hardware pre-Android-4.2, a Qualcomm-specific mechanism, named Genlock, is used. We explicitly support it. More on this below. | * On Qualcomm hardware pre-Android-4.2, a Qualcomm-specific mechanism, named Genlock, is used. We explicitly support it. More on this below. | ||
* On non-Qualcomm, pre-Android-4.2 hardware, other vendor-specific mechanisms are used, which we do not support (see e.g. {{bug|871624}}). | * On non-Qualcomm, pre-Android-4.2 hardware, other vendor-specific mechanisms are used, which we do not support (see e.g. {{bug|871624}}). | ||
| Line 104: | Line 104: | ||
Genlock is implemented in the kernel. The kernel GL driver is able to lock and unlock directly. Typically, it will place a read lock on any gralloc buffer that's bound to a texture it's sampling from, and unlock when it's done with that texture. | Genlock is implemented in the kernel. The kernel GL driver is able to lock and unlock directly. Typically, it will place a read lock on any gralloc buffer that's bound to a texture it's sampling from, and unlock when it's done with that texture. | ||
A logging patch for genlock that lets you see locking across process is here: [http://people.mozilla.com/~jmuizelaar/genlock-logging-patch] | |||
== How we lock/unlock Gralloc buffers == | == How we lock/unlock Gralloc buffers == | ||
| Line 137: | Line 139: | ||
= How Android is using Gralloc = | = How Android is using Gralloc = | ||
== EGLSurface, ANativeWindow, etc. == | == Some terminology: EGLSurface, ANativeWindow, etc. == | ||
'''EGLSurface''' is a portable EGL abstraction for a possibly multi-buffered render target. | '''EGLSurface''' is a portable EGL abstraction for a possibly multi-buffered render target. | ||
| Line 171: | Line 173: | ||
== SurfaceTexture == | == SurfaceTexture == | ||
SurfaceTexture is the server side of a client-server system, whose client side is SurfaceTextureClient. As explained above, SurfaceTextureClient is a concrete implementation of ANativeWindow. | '''SurfaceTexture''' is the server side of a client-server system, whose client side is '''SurfaceTextureClient'''. As explained above, SurfaceTextureClient is a concrete implementation of ANativeWindow. | ||
The reason to use a client-server system like this is to allow producing and compositing a surface in two different processes. | The reason to use a client-server system like this is to allow producing and compositing a surface in two different processes. | ||
Let us introduce two important functions that a client needs to call on its ANativeWindow: dequeueBuffer and queueBuffer | |||
* '''dequeueBuffer''' acquires a new buffer for the client to draw to; | |||
* '''queueBuffer''' lets the client indicate that it has finished drawing to a buffer, and queues it (e.g. for compositing). | |||
Since eglSwapBuffers internally calls dequeueBuffer and queueBuffer, this system removes the need for manual management of GraphicBuffer's as we are currently doing in our B2G code. | |||
The other benefit of this system is that most [http://en.wikipedia.org/wiki/Board_support_package BSP] vendors provide graphics profilers (e.g. Adreno Profiler from QCOM, PerfHUD ES from nVidia) which recognize the eglSwapBuffers calls as frame boundaries to collect frame-based GL information from the driver to help development and performance tuning. | |||
In Android 2, there were many buffer management systems. In Android 4, all of this is unified under SurfaceTexture. This is made possible by the great flexibility of SurfaceTexture: | |||
* SurfaceTexture supports both synchronous and asynchronous modes. | |||
* SurfaceTexture supports generic multi-buffering: it can have any number of buffers between 1 and 32. | |||
Examples: | |||
* The Codec/Camera code configures it to have several buffers (depending on hardware, for instance 9 buffers for camera preview on Unagi) and run asynchronously | |||
* SurfaceFlinger (the Android compositor) configures it to have 2--3 buffers (depending on [http://en.wikipedia.org/wiki/Board_support_package BSP]) and run synchronously. | |||
* Google Miracast uses it to encode OpenGL-rendered surfaces on-the-fly. | |||
Let us now describe how the client-server SurfaceTexture system allocates GraphicBuffer's, and how both the client and server sides keep track of these shared buffer handles. Again, SurfaceTexture is the server-side class, while SurfaceTextureClient is the client-side class. Each of them stores an array of GraphicBuffers, which is called mSlots in both classes. The GraphicBuffer objects are separate instances in SurfaceTexture and in SurfaceTextureClient, but the underlying buffer handles are the same. The mechanism here is as follows. The client side issues a SurfaceTextureClient::dequeueBuffer call to get a new buffer to paint to. If there is not already a free buffer in mSlots, and the number of buffers is under the limit (e.g. 3 for triple buffering), it sends an IPC message that results in a call to SurfaceTexture::dequeueBuffer which allocates a GraphicBuffer. After this transaction, still inside of SurfaceTextureClient::dequeueBuffer, another IPC message is sent, that results in a call to SurfaceTexture::requestBuffer to get the GraphicBuffer serialized over IPC back to it, using GraphicBuffer::flatten and GraphicBuffer::unflatten, and cache it into its own mSlots. The mSlots arrays on both sides mirror each other, so that the two sides can refer to GraphicBuffer's by index. This allows the client and server side to communicate with each other by passing only indices, without flattening/unflattening GraphicBuffers again and again. | |||
Let us now describe what happens in SurfaceTextureClient::dequeueBuffer when there is no free buffer and the number of buffers has already met the limit (e.g. 3 for triple buffering). In this case, the server side (SurfaceTexture::dequeueBuffer) will wait for a buffer to be queued, and the client side waits for that, so that SurfaceTextureClient::dequeueBuffer will not return until a buffer has actually been queued on the server side. This is what allows SurfaceTexture to support both synchronous and asynchronous modes with the same API. | |||
Let us now explain how synchronous mode works. In synchronous mode, on the client side, inside of eglSwapBuffers, when ANativeWindow::queueBuffer is called to present the frame, it sends the index of the rendered buffer to the server side. This causes the index to be queued into SurfaceTexture::mQueue for rendering. SurfaceTexture::mQueue is a wait queue for frames that want to be rendered. In synchronous mode, all the frames are shown one after another, whereas in asynchronous mode frames may be dropped. | |||
Let us now explain how SurfaceFlinger (the Android compositor) uses this system to get images onto the screen. Each time SurfaceTexture::queueBuffer is called, it causes SurfaceFlinger to start the next iteration of the render loop. In each iteration, SurfaceFlinger calls SurfaceTexture::updateTexImage to dequeue a frame from SurfaceTexture::mQueue and bind that GraphicBuffer into a texture, just like we do in GrallocTextureHostOGL::Lock. | |||
The magic part is that SurfaceFlinger does not need to do the equivalent of GrallocTextureHostOGL::Unlock. In our case, we have a separate OpenGL texture object for each TextureHost, which typically (at least in the case of a ContentHost) represent one buffer each (so a double-buffered ContentHost has two TextureHost's). So we have to unbind the GraphicBuffer from the OpenGL texture before we can hand it back to the content side --- otherwise it would remain locked for read and couldn't be locked for write for content drawing. By contrast, SurfaceFlinger does not need to worry about this because it uses only one OpenGL texture, so that when it binds a new GraphicBuffer to it for compositing, that automatically unbinds the previous one! | |||