Project Silk: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(Delete outdated information and link to proper documentation)
Line 2: Line 2:
The objective of Project Silk is to provide smoother scrolling and animation on B2G.
The objective of Project Silk is to provide smoother scrolling and animation on B2G.
To achieve the goal, it aligns HW Vsync signals among input, content painting and composition modules.
To achieve the goal, it aligns HW Vsync signals among input, content painting and composition modules.
 
Proper docs are at http://hg.mozilla.org/mozilla-central/file/b8d59286a581/gfx/doc/Silk.md
=== Architecture ===
An easier to read version is at https://github.com/changm/SilkDocs/blob/master/silk.md
<center>[[File:silk_object_diagram.png|800px|Project Silk Object relation]]</center>
<center>Fig 1. Silk object diagram.</center>
 
<center>[[File:silk_class_diagram.png|800px|Project Silk Object relation]]</center>
<center>Fig 2. Silk class diagram.</center>
 
=== VsyncDispatcherHost/Client ===
VsyncDispatcherHost/Client stands at the key position of Project Silk. The main role of this object is
# Receive HW vsync-event from vsync driver.
# Real time dispatching vsync-event to registered-event listeners.
 
There is exactly one VsyncDispatcherHostImpl object in chrome process and one VsyncDispatcherClientImpl in each content process which are derived from VsyncDispatcherHost and VsyncDispatherClient.
 
In chrome process
* VsyncDispatcherHostImpl creates a dedicated thread, named as <b>vsync dispatcher thread</b>, to dispatch vsync-event. Since vsync-event dispatching is very performance sensitive, we create a dedicate thread which only works on event dispatching to prevent transmission flow from being blocked by irrelevant tasks.
* Receive vsync events from PlatformVsyncTimer. We use HWComposer to implement the PlatformVsyncTimer in Gonk. On platforms which have no HW vsync supported, it's fallback to SW vsync driver automatically.
* Export registry functions to clients who want to receive vsync-event notification. Currently, we have three types of clients: refresh driver, compositor and input dispatcher.
* Sent vsync notifications to registered listeners, on the <b>vsync dispatcher thread</b>.
 
In Content process, VsyncDispatcherClientImpl's job is relatively simpler:
* Been created and initialized on main thread. The reason that we did not create a dedicated thread here is base on the fact: there is only one vsync listener in content side, which is nsRefreshDriver, and it can be kicked on main thread only.
* Export registry/unregistry function to user clients who want to receive vsync-event notification.
* In main thread, sent vsync notification to registered listeners.
 
==== vsync-event dispatching sequence====
<center>[[File:silk_thread_model.png|800px|Project Silk Threading model]]</center>
<center>Fig 3. Silk threading model.</center>
 
In current design,
  VsyncDispatcherHost:
    1. notify the main thread to handle input event.
    2. wait the input handling task done, then notify the compositor parent to do compose.
    3. send vsync event to content process.
    4. tick Chrome refresh driver
  VsyncDispatcherClient:
    1. tick Content refresh driver
 
 
<b>Why execute a sync InputDispatcher task on main thread instead of vsync-dispatcher thread in step 1?</b>
 
The APZ event processing code all currently runs on the main thread. APZ can't process the input at non-main thread until we complete the [https://bugzilla.mozilla.org/show_bug.cgi?id=930939 Bug 930939]. It will do the APZ off the main thread(it will update the APZ at VsyncDispatcher thread).
 
We also need to wait the APZ to update the frame matrix and forward any required input events to the child processes before we notify the compositor and refresh driver. Thus, we will have the most up to date touch coordinates rather than have a race condition where we might process the previous touch event versus the latest touch events when we composite.
 
Note1: We need to wait the APZC processed the input event in step 1, then do the remaining task. We might have a large delay for context switch.
[[File:Vsync_Threads_Context_Switch.jpg|800px|Current Thread Model]]
 
Note2: If we have [https://bugzilla.mozilla.org/show_bug.cgi?id=930939 Bug 930939], the order will be:
  VsyncDispatcherHost:
    1. APZ processs input event '''at VsyncThread'''.
    2. compositor parent to do compose.
    3. send vsync event to content process.
    4. tick Chrome refresh driver
  VsyncDispatcherClient:
    1. tick Content refresh driver
[[File:Final_Vsync_Thread_model.png|800px|Idel Thread Model]]
 
==== Object life cycle ====
VsyncDispatcherHost/Client is created as a singleton object in each process.
* VsyncDispatcherHost's life cycle in chrome process
** Create in gfxPlatform::Init
** Release in gfxPlatform::Shutdown
 
* VsyncDispatcherClient's life cycle in content process
** Create in PVsyncEventChild::Create(VsyncEvent protocol creation). Since we define PVsyncEvent as a top level protocol, VsyncDispatcherClient is constructed in the early stage of content process initialization.
** Release in PVsyncEventChild::ActorDestroy(VsyncEvent protocol destroy)
 
=== VsyncObserver ===
VsyncObserver is the base class for all vsync event observer. It will be ticked when vsync event comes. We should implement this function for our vsync-aligned task.
 
Here is the tick function prototype:
  virtual void TickTask(int64_t timestamp, uint64_t frameNumber);
 
=== VsyncEventRegistry ===
This class provide the registering interface for VsyncObserver. After registering, VsyncDispatcher will call the observer's TickTask() for vsync event. The Register() call is one-shot registry. If observer need another tick, it should register again. The Unregister() has a sync flag. All observer should call the sync unregister before they call destructor. Thus, VsyncDispatcher will not tick the observer while destroying.
 
The VsyncEventRegistry interface:
  virtual uint32_t GetObserverNum();
  virtual void Register(VsyncObserver *observer);
  virtual void Unregister(VsyncObserver *observer, bool sync);
 
=== VsyncTimerObserver ===
VsyncTimerObserver is the base class for vsync timer observer. It will be notified by PlatformVsyncTimer for vsync event. Only VsyncDispatcherHostImpl is derived from this class.
 
This class only has one interface:
  virtual void NotifyVsync(int64_t timestamp);
 
=== PlatformVsyncTimer ===
This is the platform depend vsync timer which provides the vsync events for VsyncTimerObserver.
 
The following lists the platform-dependency interfaces that should be implemented:
  virtual void Shutdown();
  virtual void Enable(bool aEnable);
  virtual uint32_t GetVsyncRate();
 
=== GonkVsyncTimer(Gonk implementation of PlatformVsyncTimer) ===
GonkVsyncTimer which is derived from PlatformVsyncTimer contains the gonk specific timer implementation. Basically, it's created in the VsyncDispatcherHost::Startup calling.
 
==== HWComposer2D ====
Here HWComposer2D is responsible to implement GonkVsyncTimer for B2G. Android HWC only supports HW Vsync after android JB. Therefore, we also need to implement a SW Vsync for system. If there is no HW Vsync support, the SW Vsync will be used. During initialization phase, VsyncDispatcher registers a Vsync notification callback into HWComposer2D and this callback will be called for every Vsync events. If there is no Vsync observer, we can disable the Vsync notifications inside HWComposer2D.
==== SW Vsync timer====
SW Vsync timer is a periodic timer which calls it's callback function with 60Hz frequency. Currently, according to the WIP patch in [https://bugzilla.mozilla.org/show_bug.cgi?id=1043822 bug 1043822], we use nanosleep() to implement the timer loop and the most parts of this timer are very similar with those in Android's SW Vsync timer. Our SW Vsync provides some APIs, such as
  void StartTimer();
  void StopTimerAsync();
  void StopTimerSync();
  double GetRate() const;
In the normal cases, EnableVsyncEvent() will use StartTimer() and StopTimerAsync() to control this timer. StopTimerSync() is called while we want to shut down VsyncDispatcher because VsyncDispatcher should wait until SW Vsync finishes all it's job.
Enabling/Disabling the Vsync module is controlled by HwComposer2D right now, and we think users shouldn't worry about which one (HW or SW Vsync) is using, so we have to use a generic API for HW Vsync and SW Vsync. For example, EnableVsync() in HwcComposer2D is the generic function which handles the enable/disable of Vsync event. We can hide the logic of the decision between HW and SW Vsync in this function. In other words, if we are using HW Vsync, EnableVsync() will call eventControl() to handle it; if we are using SW Vsync, EnableVsync() will call the APIs of SW Vsync timer. By this design, GonkDisplayJB() can also use HwcComposer2D::EnableVsync() to turn on/off the Vsync event according to the status of the display.
 
==== Power on/off screen ====
We can't control the HW Vsync event on/off during power off. We will always turn off the Vsync event at power off and force to turn on at power on. That's ok if VsyncDispatcher receive an unexpected Vsync event at power on. After receiving the Vsync event, VsyncDispatcher can just disable the HW Vsync again.
 
=== Vsync event protocol ===
PVsyncEvent, a new top level protocol, is used to pass Vsync events from chrome to content process, and also provides the register/unregister interfaces for content process to receive Vsync events. In PVsyncEventChild side, it is running at the main thread of content process. In PVsyncEventParent side, it is running at the VsyncDispatcher thread of chrome process. Inside chrome, we dispatch the Vsync event to chrome's refresh driver directly without using PVysncEvent protocol(step 4 in above dispatch sequence).
  IPC interfaces
    child:
      NotifyVsyncEvent(VsyncData aVsyncData);
    parent:
      RegisterVsyncEvent();
      UnregisterVsyncEvent();


==Porting Issue==
==Porting Issue==

Revision as of 15:37, 21 April 2015

Overview

The objective of Project Silk is to provide smoother scrolling and animation on B2G. To achieve the goal, it aligns HW Vsync signals among input, content painting and composition modules. Proper docs are at http://hg.mozilla.org/mozilla-central/file/b8d59286a581/gfx/doc/Silk.md An easier to read version is at https://github.com/changm/SilkDocs/blob/master/silk.md

Porting Issue

In silk, there are only two platform dependent objects, which are platform vsync timer and InputDispatcher.

In a word, if you want to enable project silk on a specific platform, things need to be done are

  • Constructing a concrete implementation of PlatformVsyncTimer interface
  • An InputDispatcher instance which is inherit from VsyncObserver
  • TBD: (object creation)

Testing Plan

Performace

  • Test manually
    • With automatic script or tool
    • Scroll
      • Show we get a smooth position with time
    • OMTA
      • Check b2g always composes done before next vsync
    • requestAnimationFrame
      • Check we get a better timestamp for animation

Unit test

  • VsyncDispatcher
    • Registry - vsync listener should be able to receive sync-notification steadily after registry.
    • Unregistry - vsync listener should not receive any vync notification after synchronize unregistry from VsyncDispatcher

Integration test

  • First paint - have a test case to make sure we do not break first-paint.

Benchmark

Type:

  • requestAnimationFrame, WebGL
  • OMTA test
  • scrollable layer

Project Silk Discussion at Taipei Workweek

Project Silk

Bugs

New/Open

No results.

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);


Resolved

Full Query
ID Priority Summary Status Assigned to Target milestone Last change time
930939 P1 Separate the threads that run gecko and that receive input events from the OS RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com) mozilla38 2018-10-31T13:12:35Z
980241 P2 Vsync-triggered RefreshDriver RESOLVED Jerry Shih[:jerry] (UTC+8) (inactive) --- 2014-06-12T18:07:26Z
982972 P2 investigate using RT priority in compositor on gonk RESOLVED Mason Chang [Inactive] [:mchang] --- 2014-08-15T00:11:44Z
987523 P2 Vsync-triggered CompositorParent RESOLVED Jerry Shih[:jerry] (UTC+8) (inactive) --- 2015-01-29T14:26:00Z
987527 P1 Register Vsync monitor to HWComposer RESOLVED Boris Chiou [:boris] 2.1 S3 (29aug) 2018-10-31T13:12:35Z
987529 P1 Integrate vsync dispatcher on B2G RESOLVED Jerry Shih[:jerry] (UTC+8) (inactive) 2.1 S4 (12sep) 2018-10-31T13:12:35Z
990832 P1 Build a tool to Visualize Frame Uniformity with ScrollGraph Output RESOLVED Jinho Hwang [:jeffhwang] mozilla33 2014-08-20T17:42:53Z
991420 P1 (silk) IPC Shows 1-2ms latency sending message from b2g to content process and b2g main thread to composite thread RESOLVED Mason Chang [Inactive] [:mchang] 2.0 S4 (20june) 2015-01-29T01:23:23Z
991483 -- Vsync-triggered GeckoInputDispatcher RESOLVED Jinho Hwang [:jeffhwang] --- 2018-10-31T13:12:35Z
1035076 P2 Measure vsync event latency between VsyncDispatcher and other modules in Project Silk RESOLVED Jerry Shih[:jerry] (UTC+8) (inactive) 2.1 S5 (26sep) 2015-01-29T14:49:24Z
1042017 P3 Re-Examine the APZ kinetic Scroll Friction Animation RESOLVED Mason Chang [Inactive] [:mchang] --- 2014-11-11T17:39:43Z
1043822 P2 Support SW Vsync and combine it with HW Vsync RESOLVED Boris Chiou [:boris] 2.1 S3 (29aug) 2018-10-31T13:12:35Z
1048667 P1 Implement Vsync dispatch framework RESOLVED Jerry Shih[:jerry] (UTC+8) (inactive) 2.1 S2 (15aug) 2018-10-31T13:12:35Z
1062117 -- Support the priority change for VsyncDispatcher RESOLVED Boris Chiou [:boris] --- 2018-10-31T13:12:35Z
1062331 -- Turn on by default touch event smoothing with gfx.frameuniformity.hw-vsync and gfx.touch.resample RESOLVED Mason Chang [Inactive] [:mchang] mozilla38 2018-10-31T13:12:35Z
1072627 P1 Hardware Composer Vsync Events aren't stable RESOLVED Mason Chang [Inactive] [:mchang] --- 2018-10-31T13:12:35Z
1073545 -- Add Vsync Markers to SPS Profiler RESOLVED Mason Chang [Inactive] [:mchang] mozilla35 2014-10-06T13:48:12Z
1073704 -- Input touch resampling is broken RESOLVED Michael Wu [:mwu] mozilla35 2018-10-31T13:12:35Z
1078136 -- Tune the thread priority for VsyncDispatcher thread RESOLVED --- 2018-10-31T13:12:35Z
1078152 P2 Provide performance label for frame uniformity RESOLVED Jerry Shih[:jerry] (UTC+8) (inactive) --- 2018-03-02T21:34:10Z
1078160 -- Provide an automatic testing environment for QA to check the project silk performance RESOLVED --- 2015-03-25T02:43:27Z
1078168 P3 Measure "where is my water" game overall performance with project silk RESOLVED --- 2015-12-14T06:01:05Z
1078171 P3 Measure "cut the rope" game overall performance with project silk RESOLVED --- 2015-12-14T06:00:56Z
1083530 -- Refactor GeckoTouchDispatcher to use mozilla::TimeStamp instead of nsecs_t for vsync times RESOLVED Mason Chang [Inactive] [:mchang] mozilla36 2018-10-31T13:12:35Z
1092245 -- Create a benchmark to measure refresh driver alignment to vsync and detect regressions RESOLVED Mason Chang [Inactive] [:mchang] --- 2017-01-30T16:53:10Z
1092978 P1 Integrate refresh driver for silk RESOLVED Jerry Shih[:jerry] (UTC+8) (inactive) mozilla38 2015-01-29T01:22:03Z
1094760 -- Compositor will sleep for a long time during composing if we align composing and refresh driver tick with vsync RESOLVED Jerry Shih[:jerry] (UTC+8) (inactive) --- 2015-04-08T01:46:04Z
1095242 -- Disable Vsync When Idle RESOLVED Mason Chang [Inactive] [:mchang] mozilla38 2015-02-22T14:59:55Z
1098701 -- [Silk] Gtest for project Silk RESOLVED Mason Chang [Inactive] [:mchang] mozilla38 2015-01-25T05:32:44Z
1101974 P1 Create a single VsyncDisptacher per nsBaseWidget RESOLVED Mason Chang [Inactive] [:mchang] mozilla37 2014-12-19T15:30:03Z
1102200 -- Composing time becomes much longer randomly when we unplug the usb cable on flame-kk device RESOLVED --- 2022-05-30T19:26:41Z
1102631 -- Create a software vsync timer RESOLVED Mason Chang [Inactive] [:mchang] --- 2016-07-26T21:48:15Z
1117870 -- Let the CompositorVsyncObserver unobserve vsync after a configurable number of vsyncs have passed RESOLVED Mason Chang [Inactive] [:mchang] mozilla37 2022-01-03T17:02:05Z
1118530 -- [meta] Turn on by default hardware vsync and the Vsync Aligned Compositor on b2g RESOLVED Mason Chang [Inactive] [:mchang] mozilla38 2015-07-18T14:29:51Z
1123734 -- Notify global VsyncSource in hwc RESOLVED Jerry Shih[:jerry] (UTC+8) (inactive) mozilla38 2018-10-31T13:12:35Z
1123762 -- [META] Turn on vsync-aligned refresh driver by default on b2g RESOLVED Mason Chang [Inactive] [:mchang] mozilla39 2022-06-01T22:16:00Z
1130678 -- dom/broadcastchannel/tests/test_broadcastchannel_sharedWorker.html intermittent with vsync refresh driver RESOLVED Mason Chang [Inactive] [:mchang] mozilla38 2022-06-01T22:14:38Z
1139253 -- Reuse the same thread when we start/stop vsync event for software vsync timer RESOLVED Jerry Shih[:jerry] (UTC+8) (inactive) 2.2 S8 (20mar) 2015-03-10T02:04:26Z

38 Total; 0 Open (0%); 38 Resolved (100%); 0 Verified (0%);


FAQ

Platform supporting plan
We are going to support Firefox OS first and extend this framework to other platforms after.
Can Silk improve rendering performance?
No
What version of Android is required?
We need Kit-Kat or greater. Jelly Bean and ICS are NOT supported.
Retrieved from "https://wiki.allizom.org/index.php?title=Project_Silk&oldid=1069516"