Confirmed users
137
edits
Platform/GFX/Quantum Render: Difference between revisions
m
webrender-perf try preset
(→Try pushes: Update WR syntax to an alternative that is easier to escape) |
m (webrender-perf try preset) |
||
| (19 intermediate revisions by 4 users not shown) | |||
| Line 2: | Line 2: | ||
The goal of the Quantum Render project is to take the WebRender compositor in Servo and embed it in Firefox. It will replace Gecko's existing compositor, interfacing with Gecko's main-thread layout code. As WebRender is written in Rust and uses a very different design approach, we expect to get stability and performance benefits from this switch. | The goal of the Quantum Render project is to take the WebRender compositor in Servo and embed it in Firefox. It will replace Gecko's existing compositor, interfacing with Gecko's main-thread layout code. As WebRender is written in Rust and uses a very different design approach, we expect to get stability and performance benefits from this switch. | ||
== Planning == | |||
WebRender roadmap and more detailed quarterly plans: [[Platform/GFX/WebRender_Planning|WebRender Planning]]<br /> | |||
(You'll also find links to various tracking metabugs here) | |||
== Vital stats == | == Vital stats == | ||
| Line 7: | Line 12: | ||
Canonical code repository: https://hg.mozilla.org/mozilla-central <br/> | Canonical code repository: https://hg.mozilla.org/mozilla-central <br/> | ||
Bugzilla Component: [https://bugzilla.mozilla.org/enter_bug.cgi?product=Core&component=Graphics%3A%20WebRender Core :: Graphics: WebRender] <br/> | Bugzilla Component: [https://bugzilla.mozilla.org/enter_bug.cgi?product=Core&component=Graphics%3A%20WebRender Core :: Graphics: WebRender] <br/> | ||
Triage info: [[https://wiki.mozilla.org/Platform/GFX/TriageSchedule]] | |||
Triage info: https:// | |||
Mailing list: [https://lists.mozilla.org/listinfo/dev-tech-gfx dev-tech-gfx@lists.mozilla.org] <br/> | Mailing list: [https://lists.mozilla.org/listinfo/dev-tech-gfx dev-tech-gfx@lists.mozilla.org] <br/> | ||
IRC channel: #gfx <br/> | IRC channel: #gfx <br/> | ||
Project owner: Maire Reavy <br/> | Project owner: Maire Reavy <br/> | ||
Where is it on: [[Platform/GFX/WebRender_Where]] | |||
== Getting the code == | == Getting the code == | ||
| Line 60: | Line 26: | ||
== Build instructions == | == Build instructions == | ||
Building is the same as building a regular mozilla-central build | Building is the same as building a regular mozilla-central build. However, WebRender may be disabled by default at runtime if you are running on hardware that we have not whitelisted. You can enable it one of three ways: | ||
* | * Manually set the <tt>gfx.webrender.enabled</tt> or <tt>gfx.webrender.all</tt> pref to true. You can do this from about:config, and it requires restarting the browser to take effect. | ||
* Run firefox with <tt>MOZ_WEBRENDER=1</tt> as an environment variable. This will attempt to enable WebRender at runtime. | * Run firefox with <tt>MOZ_WEBRENDER=1</tt> as an environment variable. This will attempt to enable WebRender at runtime. | ||
| Line 74: | Line 39: | ||
cargo build --features=capture,replay,pathfinder | cargo build --features=capture,replay,pathfinder | ||
If you use mozilla-central, you | If you use mozilla-central (recommended), you should use the normal Gecko workflow and submit patches for review via Phabricator. If you really want, you can also submit PRs against the <tt>servo/webrender</tt> repository on Github, and one of the core contributors will review and merge your patch into mozilla-central. The mozilla-central copy of webrender is considered canonical, and there is a periodic one-way sync from mozilla-central to Github. | ||
=== Testing third-party rust library changes === | === Testing third-party rust library changes === | ||
| Line 113: | Line 69: | ||
=== On Gecko integration === | === On Gecko integration === | ||
If you are familiar with the way Gecko normally works, the Quantum Render changes should be relatively straightforward. Instead of using a ClientLayerManager, we create a [https://hg.mozilla.org/projects/graphics/file/tip/gfx/layers/wr/WebRenderLayerManager.cpp WebRenderLayerManager] instance. However, this "layer manager" doesn't actually manage layers, because WebRender doesn't use layers. Instead, it walks the Gecko display list and calls the <tt>BuildWebRenderCommands</tt> function on each display item to generate WebRender display items. This display list is sent over the [https://hg.mozilla.org/projects/graphics/file/tip/gfx/layers/ipc/PWebRenderBridge.ipdl PWebRenderBridge] IPDL channel, which is conceptually similar to PLayerTransaction in Gecko. The commands are received in [https://hg.mozilla.org/projects/graphics/file/tip/gfx/layers/wr/WebRenderBridgeParent.cpp WebRenderBridgeParent] which interprets the messages, and talks to the core WebRender library using the API in [https://hg.mozilla.org/projects/graphics/file/tip/gfx/webrender_bindings/webrender_ffi.h webrender_ffi.h] (via various wrapper abstractions in <tt>gfx/webrender_bindings/</tt>). The API is implemented in the [https://hg.mozilla.org/projects/graphics/file/tip/gfx/webrender_bindings/src/bindings.rs webrender_bindings] crate, which is written in Rust. | |||
If you are familiar with the way Gecko normally works, the Quantum Render changes should be relatively straightforward. Instead of using a ClientLayerManager, we create a [https://hg.mozilla.org/projects/graphics/file/tip/gfx/layers/wr/WebRenderLayerManager.cpp WebRenderLayerManager] instance. | |||
Look through [https://bugzilla.mozilla.org/buglist.cgi?component=Graphics%3A%20WebRender&product=Core&bug_status=__open__&list_id=13349798 open unassigned bugs] (with no open dependencies) in the WebRender bugzilla component to find things to work on. | Look through [https://bugzilla.mozilla.org/buglist.cgi?component=Graphics%3A%20WebRender&product=Core&bug_status=__open__&list_id=13349798 open unassigned bugs] (with no open dependencies) in the WebRender bugzilla component to find things to work on. | ||
| Line 121: | Line 75: | ||
=== On reftests === | === On reftests === | ||
Many of the existing gecko reftests (over 13000 of them) are passing with Quantum Render. However, there are still around | Many of the existing gecko reftests (over 13000 of them) are passing with Quantum Render. However, there are still around 80 failing reftests. These are listed in [https://docs.google.com/spreadsheets/d/1DcqaMvt3I38HIl3-RDS_6aiX67nQWItNcwmVt5utbDk/edit?usp=sharing this Google sheet]. You're welcome to work on getting these passing - please follow the procedure below: | ||
* Identify a single reftest you wish to work on. Try to pick one that sounds unrelated to other reftests that people are working on, because often one patch will fix multiple related reftests. | * Identify a single reftest you wish to work on. Try to pick one that sounds unrelated to other reftests that people are working on, because often one patch will fix multiple related reftests. | ||
* File a bug for that reftest, and mark it as blocking {{bug|1322815}} (webrender-reftests). | * File a bug for that reftest, and mark it as blocking {{bug|1322815}} (webrender-reftests). | ||
* Work on the fix. The best way to do this is to remove the fails-if or skip-if annotation in your local checkout and run the reftest using <tt>mach reftest</tt>. Debug and fix as needed | * Work on the fix. The best way to do this is to remove the fails-if or skip-if annotation in your local checkout and run the reftest using <tt>mach reftest --enable-webrender</tt>. Debug and fix as needed. | ||
* | * You should do at least one try push after writing your fix to verify it works in automation, as well as to identify any other tests that are fixed by your patch. Make sure to update the annotations for all newly-passing tests as part of your final commit. Try not to introduce regressions, although breaking a small number of tests to make a larger number pass might be acceptable. | ||
* Land your fix after getting review as appropriate. | * Land your fix after getting review as appropriate. | ||
| Line 134: | Line 88: | ||
=== Locally === | === Locally === | ||
If you want to run QR tests locally, you can use <tt>mach</tt> to run them as you would for Gecko normally. Just be sure to | If you want to run QR tests locally, you can use <tt>mach</tt> to run them as you would for Gecko normally. Just be sure to pass <tt>--enable-webrender</tt> to the <tt>mach</tt> command when running to enable WebRender. For example, to run the sanity reftests which is a good smoketest that you didn't horribly break everything, you can do this: | ||
./mach reftest --enable-webrender layout/reftests/reftest-sanity/ | |||
You can also run the WR standalone tests (without the Gecko stuff) locally. The page at https://developer.mozilla.org/en-US/docs/Mozilla/QA/WebRender describes how to do this. | You can also run the WR standalone tests (without the Gecko stuff) locally. The page at https://developer.mozilla.org/en-US/docs/Mozilla/QA/WebRender describes how to do this. | ||
| Line 142: | Line 96: | ||
=== Try pushes === | === Try pushes === | ||
You can (and are encouraged to) push changes to tryserver to test them out before landing them. For linux64, the tests will run using a regular linux64 build. However, you need to specify <tt>linux64-qr</tt> as the test platform to have QR enabled during testing. The same goes for windows10-64. So, for example, to run all the available tests on QuantumRender-enabled builds, you could use this trychooser syntax: | You can (and are encouraged to) push changes to tryserver to test them out before landing them. | ||
The simplest way to render webrender-related tests is to use the "webrender" try preset: | |||
./mach try --preset webrender | |||
This preset runs most tests relevant to webrender with the exception of some tests running on Pixel 2 devices (for now). It is also possible to push to try using try or try fuzzy syntax explicitly: | |||
For linux64, the tests will run using a regular linux64 build. However, you need to specify <tt>linux64-qr</tt> as the test platform to have QR enabled during testing. The same goes for <tt>windows10-64</tt>/<tt>windows10-64-qr</tt> and <tt>macosx64</tt>/<tt>macosx64-qr</tt>. So, for example, to run all the available desktop tests on QuantumRender-enabled builds, you could use this trychooser syntax: | |||
try: -b do -p linux64,win64,macosx64 -u all[linux64-qr,windows10-64-qr,macosx64-qr] -t all[linux64-qr,windows10-64-qr,macosx64-qr] | try: -b do -p linux64,win64,macosx64 -u all[linux64-qr,windows10-64-qr,macosx64-qr] -t all[linux64-qr,windows10-64-qr,macosx64-qr] | ||
Note that the query above does not run wrench tests. | |||
There is also a webrender-perf try preset which only runs some of the talos tests relevant to webrender: | |||
./mach try --preset webrender-perf | |||
Running Android tests is a little tricker, because we only run Gecko reftests on Pixel 2 devices for Android, and we don't have a lot of those so there's often a backlog of jobs for them. In order to minimize the chance of people accidentally queuing up jobs on these device that they don't actually need, jobs that run on these devices are not selectable by default. Instead you have to run `./mach try fuzzy --full` which lists all the "hidden" jobs, and explicitly select them. From the command line, you can select the Android QuantumRender jobs like so: | |||
./mach try fuzzy --full -q "'"'-qr android !fis' # !fis excludes fission variants which also show up with --full but are perma-fail | |||
You can also run WebRender's standalone tests on tryserver (linting and Linux/Windows tests only for now, macOS is tracked in {{bug|1516568}}). As trychooser syntax is deprecated, the way to run these tests is to select them in <tt>./mach try fuzzy</tt> with the query string <tt>^webrender-</tt>. You can do this on the command line like so: | You can also run WebRender's standalone tests on tryserver (linting and Linux/Windows tests only for now, macOS is tracked in {{bug|1516568}}). As trychooser syntax is deprecated, the way to run these tests is to select them in <tt>./mach try fuzzy</tt> with the query string <tt>^webrender-</tt>. You can do this on the command line like so: | ||
./mach try fuzzy -q '^webrender-' | ./mach try fuzzy -q '^webrender-' | ||
To run all of the QuantumRender tests as well as the WebRender standalone tests, you can use the following command: | To run all of the desktop QuantumRender tests as well as the WebRender standalone tests, you can use the following command: | ||
./mach try fuzzy -q "'-qr" -q '^webrender-' | ./mach try fuzzy -q "'-qr" -q '^webrender-' | ||
or without talos/raptor: | or without talos/raptor: | ||
./mach try fuzzy -q "'"'-qr !talos !raptor' -q '^webrender-' | ./mach try fuzzy -q "'"'-qr !talos !raptor' -q '^webrender-' | ||
If you have a particular set of tests you run frequently that cannot be represented by trychooser syntax, the best way to do it is this: | If you have a particular set of tests you run frequently that cannot be easily represented by trychooser or fuzzy syntax, the best way to do it is this: | ||
* First, do a push using <tt>./mach try fuzzy</tt> where you select the set of jobs you care about | * First, do a push using <tt>./mach try fuzzy</tt> where you select the set of jobs you care about | ||
* The head commit of that try push will have a <tt>try_tasks_config.json</tt> file which lists the jobs. Copy that file into a local commit | * The head commit of that try push will have a <tt>try_tasks_config.json</tt> file which lists the jobs. Copy that file into a local commit | ||
| Line 178: | Line 148: | ||
to build and install. You will need to enable WR by setting <tt>gfx.webrender.all</tt> to true either via <tt>about:config</tt> or in a [[Platform/Platform-specific_build_defines#Prefs_files|prefs file]] that is used by GeckoView. If you change any Rust/C++ code, you need to re-run <tt>./mach build binaries && ./mach package && ./mach android install-geckoview_example</tt> to get the updates on-device. | to build and install. You will need to enable WR by setting <tt>gfx.webrender.all</tt> to true either via <tt>about:config</tt> or in a [[Platform/Platform-specific_build_defines#Prefs_files|prefs file]] that is used by GeckoView. If you change any Rust/C++ code, you need to re-run <tt>./mach build binaries && ./mach package && ./mach android install-geckoview_example</tt> to get the updates on-device. | ||
=== Debugging === | === Debugging with Android Studio === | ||
The Android Studio instructions on the [https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/Simple_Firefox_for_Android_build#Developing_Firefox_for_Android_in_Android_Studio build instructions] page work for the GeckoView Example App, as long as you choose the "geckoview_example" run configuration instead of the "app" run configuration. Debugging of Java and C++ code works in both the parent process and the content process, if you add this to your <tt>~/.lldbinit</tt> file: | The Android Studio instructions on the [https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/Simple_Firefox_for_Android_build#Developing_Firefox_for_Android_in_Android_Studio build instructions] page work for the GeckoView Example App, as long as you choose the "geckoview_example" run configuration instead of the "app" run configuration. Debugging of Java and C++ code works in both the parent process and the content process, if you add this to your <tt>~/.lldbinit</tt> file: | ||
| Line 188: | Line 158: | ||
Note that to debug a content process, you have to attach to it explicitly. After starting the app in a debug configuration from Android Studio, click on the "Attach debugger to Android process" icon (tall rectangle with a bug on it) in the toolbar and select the process you want to attach to. You can be attached to multiple processes at the same time. | Note that to debug a content process, you have to attach to it explicitly. After starting the app in a debug configuration from Android Studio, click on the "Attach debugger to Android process" icon (tall rectangle with a bug on it) in the toolbar and select the process you want to attach to. You can be attached to multiple processes at the same time. | ||
=== Debugging with the WebRender Debugger === | |||
WebRender has a built-in debug server that can be interacted with a browser-based frontend. This debugger can let you toggle debug flags in WebRender, as well as extract some of WR's internal data structures "live". The browser-based frontend can be opened by pointing your browser to the <tt>gfx/wr/debugger/index.html</tt> file, and navigating the different panes will give you an idea of what kinds of things it can do. | |||
The debug server in WebRender is not built or enabled by default. To build it, you need to add | |||
ac_add_options --enable-webrender-debugger | |||
to your mozconfig file. And to enable it, you need to set the <tt>gfx.webrender.start-debug-server</tt> pref to true. All WR renderer instances created after this pref is enabled will start a separate instance of the debug server. On desktop, this means new browser windows opened after the pref is enabled will each have a debug server, although in practice only the first will be able to acquire the port and the rest will fail. On mobile (for example in the GVE) the pref needs to be set during startup for the WR renderer instance to start the debug server. | |||
Once you have the debug server enabled, you need to ensure the frontend can talk to it. If you are running the frontend and WR on the same machine this should just work. For debugging WR on Android, you will generally load the frontend on a desktop, and will need to use <tt>adb forward tcp:3583 tcp:3583</tt> to port-forward the connection to the device being debugged. Then click on the "Connect" button in the frontend and you should be good to go. | |||
=== Bisecting === | === Bisecting === | ||
| Line 197: | Line 177: | ||
=== Captures === | === Captures === | ||
It is possible to get WR captures from GeckoView-based products that have WR enabled. To trigger the capture, you need to run the product (e.g. GVE) with the <tt>devtools.remote.usb.enabled</tt> pref enabled, and use <tt>Web Developer > WebIDE</tt> in Firefox desktop to connect to the phone. In the "Main Process" browser console, run this command: | |||
window.windowUtils.wrCapture(); | window.windowUtils.wrCapture(); | ||