Platform/GFX/Quantum Render

From MozillaWiki
< Platform‎ | GFX
Revision as of 19:47, 16 October 2018 by Bholley (talk | contribs) (Add mozilla-central contribution workflow)
Jump to navigation Jump to search

What is it

The goal of the Quantum Render project is to take the WebRender compositor in Servo and embed it in Firefox. It will replace Gecko's existing compositor, interfacing with Gecko's main-thread layout code. As WebRender is written in Rust and uses a very different design approach, we expect to get stability and performance benefits from this switch.

Vital stats

Canonical code repository: https://hg.mozilla.org/mozilla-central
Bugzilla Component: Core :: Graphics: WebRender
Tracking metabugs:

General prioritization metric for our stage-wr-trains bug (bug 1386669) is:

  • P1: block "riding into beta"
    • very important - looking to find owners and land fixes asap
    • could be blocking other WR development or solid use of the feature
    • can't get into beta with these unresolved.
  • P2 & P3: block "riding into release"
    • not blocking other developers or solid use of the feature, but it shows a regression or correctness problem that we can't ship with.
    • We can ride into beta with these, but they need to be fixed before we ride-to-release
    • P2s should be the list developers look at to find their next WR bug
    • If there are no currently unassigned P2 bugs, ping mreavy in #gfx on irc
  • P4: needs a call to determine if they should block MVP (we are typically waiting on info to make a decision)

Meta bugs for follow-on work:

Triage info: https://gist.github.com/Gankro/b05d5bb8c91faff6d23e1d2549e26fe4

Mailing list: dev-tech-gfx@lists.mozilla.org
IRC channel: #gfx
Project owner: Maire Reavy

Getting the code

The work for the Quantum Render project is in mozilla-central and uses the normal gecko development workflow.

Build instructions

Building is the same as building a regular mozilla-central build. In fact, any regular mozilla-central build should have WebRender built in, unless you explicitly disable it via ac_add_options --disable-webrender. However, WebRender will be disabled by default at runtime. You can enable it one of three ways:

  • Add ac_add_options --enable-webrender to your mozconfig. All this does is build with the gfx.webrender.enabled pref turned on by default instead of turned off by default.
  • Manually flip the gfx.webrender.enabled pref. You can do this from about:config, and it requires restarting the browser to take effect.
  • Run firefox with MOZ_WEBRENDER=1 as an environment variable. This will attempt to enable WebRender at runtime.

Note: WebRender may still be disabled by other runtime conditions. Common conditions include a disabled GPU process (on Windows) or disabled hardware acceleration. You can check the WebRender status by going to about:support and looking at the WebRender line in the graphics section. On Linux hardware acceleration is disabled by default, so set layers.acceleration.force-enabled to true in about:config (restart required), or run with MOZ_ACCELERATED=1 in the environment to ensure HWA is enabled and doesn't block WebRender.

Contribution workflow

You can hack on WebRender either via a checkout of mozilla-central or via a checkout of the webrender repository. Using mozilla-central allows you to easily build/test Firefox and trigger try pushes, especially when you have interdependent changes across Firefox and WebRender. You can also build standalone WebRender from within mozilla-central, via i.e. 

 cd gfx/webrender
 cargo build --features=capture,replay,pathfinder

If you use mozilla-central, you have the option to request review via Phabricator, but your final changes must not land directly, and should be submitted to the standalone WebRender repository. Git users can do this like so: 

 cd /path/to/m-c
 git format-patch -pk --relative=gfx BASE_REVISION gfx/webrender gfx/webrender_api
 cd /path/to/webrender
 git checkout -b my_branch
 git am /path/to/m-c/0*
 git push my_branch

You should then ping a bors operator to carry your apply your r+ to the GitHub PR. Once your change lands in upstream WebRender, it will be synced to mozilla-central within a day or two.

Testing third-party rust library changes

Sometimes when hacking on Quantum Render, you'll need to make a change to one of the upstream dependencies of the webrender library (say for example euclid). However, you may need to test out your changes in the QR build. The way to do this is not obvious, because the QR build uses the vendored copy of euclid (in third_party/rust/euclid) which you can't directly modify without violating the checksum checks. Instead, what you need to do is this:

In $MOZILLA_ROOT/Cargo.toml, there should be a section at the bottom called [patch.crates-io]. Add an entry like so: 

 [patch.crates-io]
 ...
 euclid = { path = "third_party/rust/euclid" }

Then run: 

 ./mach vendor rust

Hacking

There are lots of places to help hack on Quantum Render. See the sections below for some info on where to get started.

On WebRender

A good overview of WebRender and what it is can be found in this blog post. WebRender is developed as a standalone library in the WebRender github repo. There is a wiki with several informative documents on the architecture of WebRender. In particular, Debugging WebRender document can be useful to get familiar with the tools and tricks regularly employed by developers. There is also a tutorial-style issue 3070 describing a step-by-step process of investigating and fixing a particular rendering issue.

Look through the issues list to find things to work on. Some WebRender bugs that are good for new contributors are tagged on GitHub:

On Gecko integration

(Note: the description below applies to the "layers mode" integration with WebRender, which is deprecated now. We are in the process of transitioning to the "layers-free mode", where Gecko display items are directly converted to WebRender commands via their CreateWebRenderCommands functions. Some of the content below still applies though; in particular we still create a WebRenderLayerManager, which manages the conversion from Gecko display items to WebRender display items.)

If you are familiar with the way Gecko normally works, the Quantum Render changes should be relatively straightforward. Instead of using a ClientLayerManager, we create a WebRenderLayerManager instance. This layer manager creates its own layer instances, and when those layers have their Render() function called, they generate WebRender-specific commands. These commands are sent over the PWebRenderBridge IPDL channel, which is conceptually similar to PLayerTransaction in Gecko. The commands are received in WebRenderBridgeParent which interprets the messages, and talks to the core WebRender library using the API in webrender_ffi.h (via various wrapper abstractions in gfx/webrender_bindings/). The API is implemented in the webrender_bindings crate, which is written in Rust.

Look through open unassigned bugs (with no open dependencies) in the WebRender bugzilla component to find things to work on.

On reftests

Many of the existing gecko reftests (over 13000 of them) are passing with Quantum Render. However, there are still around 140 failing reftests. These are listed in this Google sheet. You're welcome to work on getting these passing - please follow the procedure below:

  • Identify a single reftest you wish to work on. Try to pick one that sounds unrelated to other reftests that people are working on, because often one patch will fix multiple related reftests.
  • File a bug for that reftest, and mark it as blocking bug 1322815 (webrender-reftests).
  • Work on the fix. The best way to do this is to remove the fails-if or skip-if annotation in your local checkout and run the reftest using mach reftest. Debug and fix as needed. Note that you can likely work on the bug on Windows/OS X locally, but we only run the tests on Linux64 in automation. So for best results, prefer working on Linux locally when possible.
  • If you need to do try pushes, use syntax try: -b do -p linux64 -u all[linux64-qr] -t none. You should do at least one try push after writing your fix to verify it works in automation, as well as to identify any other tests that are fixed by your patch. Make sure to update the annotations for all newly-passing tests as part of your final commit. Try not to introduce regressions, although breaking a small number of tests to make a larger number pass might be acceptable.
  • Land your fix after getting review as appropriate.

In general, tests that are marked skip-if(webrender) are the worst, because they cause the reftest run to crash or hang and prevent other reftests from running. Slightly better than this are the random-if(webrender) tests - these can intermittently fail or pass, and so mean that there is a race condition somewhere. Slighly better than this are the fail-if(webrender) tests - these fail, but at least they do so consistently. Best of all are the ones with no webrender annotation at all, which means they behave the same as a Gecko build would.

Testing

Locally

If you want to run tests locally, you can use mach to run them as you would for Gecko normally. Just be sure to set MOZ_WEBRENDER=1 when running to enable WebRender. On Linux, you should also set MOZ_ACCELERATED=1 to enable hardware acceleration, without which WebRender will be disabled at runtime. For example, to run the sanity reftests which is a good smoketest that you didn't horribly break everything, you can do this: 

 MOZ_ACCELERATED=1 MOZ_WEBRENDER=1 ./mach reftest layout/reftests/reftest-sanity/

Try syntax

You can (and are encouraged to) push changes to tryserver to test them out before landing them. For linux64, the tests will run using a regular linux64 build. However, you need to specify linux64-qr as the test platform to have QR enabled during testing. The same goes for windows10-64. So, for example, to run all the available tests on QuantumRender-enabled builds, you could use this trychooser syntax: 

 try: -b do -p linux64,win64,macosx64 -u all[linux64-qr,windows10-64-qr,macosx64-qr] -t all[linux64-qr,windows10-64-qr,macosx64-qr]

Automation

The integration branches (inbound and autoland) run a subset of QuantumRender tests on every push. The mozilla-central branch runs all that, plus a few extra tests that are not yet marked as tier-1.

Further reading

There is more information at the following locations:

Retrieved from "https://wiki.allizom.org/index.php?title=Platform/GFX/Quantum_Render&oldid=1202498"