Platform/GFX/Quantum Render

What is it

The goal of the Quantum Render project is to take the WebRender compositor in Servo and embed it in Firefox. It will replace Gecko's existing compositor, interfacing with Gecko's main-thread layout code. As WebRender is written in Rust and uses a very different design approach, we expect to get stability and performance benefits from this switch.

Vital stats

Canonical code repository: https://hg.mozilla.org/projects/graphics
Git mirror: https://github.com/mozilla/gecko-projects/tree/graphics
TreeHerder: https://treeherder.mozilla.org/#/jobs?repo=graphics
Bugzilla Component: Core :: Graphics: WebRender
Tracking metabug: bug 1311790
Mailing list: dev-tech-gfx@lists.mozilla.org
IRC channel: #gfx
Project owner: Milan Srekovic

Getting the code

The work for the Quantum Render project generally lands on the graphics branch. However, mozilla-central is merged into graphics almost daily, and graphics is merged back into mozilla-central roughly every 7-10 days, so even if you use m-c it should be pretty close. The instructions below assume you want to use the graphics branch.

Using mercurial

To clone the repo from scratch, run this:

 hg clone https://hg.mozilla.org/projects/graphics

Or if you have an existing mozilla repo, add this to your .hgrc file:

 [paths]
 graphics = ssh://hg.mozilla.org/projects/graphics/

and pull using hg pull graphics.

Using git

See the README at https://github.com/mozilla/gecko-projects, use "graphics" as the BRANCH

Build instructions

Building is the same as building a regular mozilla-central build. In fact, any regular mozilla-central build should have WebRender built in, unless you explicitly disable it via ac_add_options --disable-webrender. However, WebRender will be disabled by default at runtime. You can enable it one of three ways:

• Add ac_add_options --enable-webrender to your mozconfig. All this does is build with the gfx.webrender.enabled pref turned on by default instead of turned off by default.
• Manually flip the gfx.webrender.enabled pref. You can do this from about:config, and it requires restarting the browser to take effect.
• Run firefox with MOZ_WEBRENDER=1 as an environment variable. This will attempt to enable WebRender at runtime.

Note that WebRender may still be disabled by other runtime conditions (e.g. on Windows if you don't have the GPU process, WebRender will be disabled). You can check the WebRender status by going to about:support and looking at the WebRender line in the graphics section.

Testing third-party rust library changes

Sometimes when hacking on Quantum Render, you'll need to make a change to one of the upstream dependencies of the webrender library (say for example euclid). However, you may need to test out your changes in the QR build. The way to do this is not obvious, because the QR build uses the vendored copy of euclid (in third_party/rust/euclid) which you can't directly modify without violating the checksum checks. Instead, what you need to do is this:

• First we make a copy of the vendored euclid repo

 cd $MOZILLA_ROOT/third_party/rust
 cp -R euclid euclid_modified

• Then, make your desired changes to euclid_modified.
• Then, we need to tell the build system to use euclid_modified instead of euclid. Note that for this to work, the version numbers in the Cargo.toml files should be the same in euclid and euclid_modified. The following steps add a [replace] section to the top-level libgkrust Cargo.toml file to use the modified euclid. See http://doc.crates.io/manifest.html#the-replace-section for details. You'll need to replace the 0.10.3 in the command below with whatever the library version in (as of this writing euclid is at v0.10.3 in the graphics repo).

 cd $MOZILLA_ROOT/toolkit/library/rust/
 echo -e '[replace]\n"euclid:0.10.3" = { path = "../../../third_party/rust/euclid_modified" }" >> Cargo.toml
 cargo update -p euclid # This updates Cargo.lock

• And then build with your modified euclid:

 cd $MOZILLA_ROOT
 mach build

Hacking

There are lots of places to help hack on Quantum Render. See the sections below for some info on where to get started.

On WebRender

A good overview of WebRender and what it is can be found in this blog post. WebRender is developed as a standalone library in the WebRender github repo. Look through the issues list to find things to work on. Some WebRender bugs that are good for new contributors are tagged on GitHub:

• WebRender's “easy” bugs do not require specific experience with graphics rendering or WebRender.
• WebRender's “less-easy” bugs assume familiarity with Rust, graphics, and WebRender.

On Gecko integration

If you are familiar with the way Gecko normally works, the Quantum Render changes should be relatively straightforward. Instead of using a ClientLayerManager, we create a WebRenderLayerManager instance. This layer manager creates its own layer instances, and when those layers have their Render() function called, they generate WebRender-specific commands. These commands are sent over the PWebRenderBridge IPDL channel, which is conceptually similar to PLayerTransaction in Gecko. The commands are received in WebRenderBridgeParent which interprets the messages, and talks to the core WebRender library using the API in webrender_ffi.h (via various wrapper abstractions in gfx/webrender_bindings/). The API is implemented in the webrender_bindings crate, which is written in Rust.

Look through open unassigned bugs (with no open dependencies) in the WebRender bugzilla component to find things to work on.

On reftests

Many of the existing gecko reftests (over 13000 of them) are passing with Quantum Render. However, there are still around 150 failing reftests. These are listed in this Google sheet. You're welcome to work on getting these passing - please follow the procedure below:

• Identify a single reftest you wish to work on. Try to pick one that sounds unrelated to other reftests that people are working on, because often one patch will fix multiple related reftests.
• File a bug for that reftest, and mark it as blocking bug 1322815 (webrender-reftests).
• Update the Google sheet with the bug number and your name.
• Work on the fix. The best way to do this is to remove the fails-if or skip-if annotation in your local checkout and run the reftest using mach reftest. Debug and fix as needed. Note that you can likely work on the bug on Windows/OS X locally, but we only run the tests on Linux64 in automation. So for best results, prefer working on Linux locally when possible.
• If you need to do try pushes, use syntax try: -b do -p linux64 -u all[linux64-qr] -t none. You should do at least one try push after writing your fix to verify it works in automation, as well as to identify any other tests that are fixed by your patch. Make sure to update the annotations for all newly-passing tests as part of your final commit. Try not to introduce regressions, although breaking a small number of tests to make a larger number pass might be acceptable.
• Land your fix after getting review as appropriate.

In general, tests that are marked skip-if(webrender) are the worst, because they cause the reftest run to crash or hang and prevent other reftests from running. Slightly better than this are the random-if(webrender) tests - these can intermittently fail or pass, and so mean that there is a race condition somewhere. Slighly better than this are the fail-if(webrender) tests - these fail, but at least they do so consistently. Best of all are the ones with no webrender annotation at all, which means they behave the same as a Gecko build would.

Committing changes

The process for making changes is very similar to that for integration branches like inbound. File a bug in the WebRender component, attach patches, get review, push to the graphics repo. The bug will be marked as RESOLVED FIXED once the graphics repo is merged back to m-c (which happens roughly every 7-10 days).

Note that there is also a dummy bugzilla account for the entire graphics team (search for ":gfx"). Flagging this account as the reviewer indicates that you don't care who does the review, and anybody in the graphics team can do it. Members of the graphics team are expected to have the :gfx account on their user watchlist so they will see review requests to the team. The list of pending review requests can also be seen in this Bugzilla query.

Try syntax

You can (and are encouraged to) push changes to tryserver to test them out before landing them in the graphics branch. For linux64, the tests will run using a regular linux64 build. However, you need to specify linux64-qr as the test platform to have QR enabled during testing. So, for example, to run all the available tests on linux64 QuantumRender-enabled builds, you could use this trychooser syntax:

 try: -b do -p linux64 -u all[linux64-qr] -t none

Note here that the build platform is linux64 but the test platform is linux64-qr.

Note also that tests are currently only running on linux64-qr, so make sure to include that test platform if you want test coverage.

Sheriffing

Currently the graphics repo is self-managed - that is, the graphics team members are responsible for doing their own sheriffing - this includes starring failures, backing out of bad commits, doing merges, etc. Currently the graphics branch is being merged to m-c roughly every 7-10 days; however, we are doing almost-daily merges from m-c to the graphics branch (see section below). In particular this means that QuantumRender-specific changes should land on the graphics branch and not on other integration branches.

We do have a pulsebot instance watching commits to the graphics repo, and it will treat the graphics branch the same as it does the other integration branches. That is, it will comment in the bug when commits are landed, but will not close the bug. The bug will be resolved once the change is merged to mozilla-central. Pulsebot will also report commits to #gfx.

Starring failures

In general we want to try and keep zero unstarred failures on the graphics branch. Most failures are pre-existing intermittents, but we should star them anyway in order to make sure we didn't introduce new failures or make something perma-fail. Oftentimes there will be random one-off failures or new failures that come in from m-c which we can mostly ignore (but still need to star). Starring failures is basically a black art - if in doubt about what to do with a failure, ask one of the m-c sheriffs or ping kats in #gfx who is currently doing most of the sheriff stuff.

Merging from mozilla-central

The following commands (on a m-c mercurial clone which has the graphics repo as an additional upstream path) will do a merge from m-c:

  hg update graphics       # switch to graphics branch
  hg pull -u graphics      # get latest graphics changesets
  hg pull central          # get latest m-c changesets
  hg merge central         # perform a merge. you may get prompted to resolve conflicts in this step
  hg commit -m "Merge m-c to graphics, a=merge"  # commit the merge
  mach build               # ensure that the merge builds properly, make additional commits as needed
  mach reftest layout/reftests/reftest-sanity # Run sanity reftests just to make sure the build runs
  hg push graphics -r tip  # push the merge

Automation

The regular mozilla-central branches (mozilla-central, inbound, and autoland) run a subset of QuantumRender tests on every push. The graphics branch runs all that, plus a few extra tests that are not yet marked as tier-1.

Further reading

There is more information at the following locations:

• https://public.etherpad-mozilla.org/p/gfx