Oxidation

Oxidation is a project to integrate Rust code in and around Firefox.

Rust support has been required on all platforms since Firefox 54, and the first major Rust components were shipped in Firefox 56 (encoding_rs) and 57 (Stylo). Moving forward, the goal of Oxidation is to make it easier and more pleasant to use Rust in Firefox, and correspondingly to increase the amount of Rust code in Firefox.

This page is intended to serve as the starting point for all matters relating to Rust code in Firefox: the what, the why, and the how.

Guidelines

The goal of this section is to provide some high-level guidelines about when Rust should be used.

In summary, Rust should be used in the following situations.

• For new components and completely rewritten components there should be a strong bias towards using Rust, especially for code around Firefox but not within Firefox.
• For existing components it's more complicated!

The following sections have more detail. Ultimately, choice of language for a code component is an engineering decision, with corresponding trade-offs, and is best decided by individual teams.

Rust Strengths

Rust has the following strengths.

• Memory safety, which prevents crashes and security vulnerabilities.
• Thread safety, which enables improved performance via parallelism.
• Nimbleness: the safety makes it easy to make significant changes quickly and with confidence.
• Pleasant to use, particularly once a moderate level of experience has been reached.
• A great community.

Rust Weaknesses

One major issue with Rust relates to personnel.

• There is a wide variety of experience levels within Mozilla, for both coding and reviewing.
• Rust's learning curve is steep at the start, which can be intimidating.

There are also technical challenges.

• Compilation is slow.
• Crossing the C++/Rust boundary can be difficult.

See "Blockers and obstacles" below for more details about work being done to remedy these weaknesses.

Recommendations

Therefore, Rust is most suitable in the following situations.

• For components that are relatively standalone, with small and simple APIs.
  • This minimizes the C++/Rust boundary layer issues.
  • Infrastructure tools that are standalone programs are ideal.
  • Note that it's good software engineering practice to write loosely-coupled components anyway.
• For components that process untrusted input, e.g. parsers.
  • Rust's memory safety is a big help here.
  • See the "Writing parsers like it is 2017" paper for lots of good details.
• For components where parallelism can provide big performance wins.
• For components where Servo has demonstrated success.

In terms of where to keep Rust crates, there are three options.

• Put the crate in mozilla-central or in Servo's repository.
  • For binding code, the decision to put it into Gecko or Servo can be difficult. The best choice depends on the details of the binding code in question.
• Put the crate on crates.io and use Cargo to access it at build-time.
  • This is only suitable for highly general-purpose crates, such as smallvec.
• Put the crate somewhere else (e.g. a separate GitHub repository), and regularly vendor it into mozilla-central.
  • This makes sense for pre-existing third-party crates that we choose to import.
  • Otherwise, this option is not recommended, because vendoring is something of a hassle.

In general, erring on the side of tighter coupling is advisable. For example, the heapsize crate used in memory reporting was moved to crates.io, and then other crates came to depend on it. Later on it needed major API changes, and we ended up replacing it with a new crate called malloc_size_of (stored in Servo's repository) because that was easier than modifying heapsize.

Documentation and assistance

Rust in general

• The Rust Documentation page is the best place to start. In particular, the The Rust Programming Language provides a good overview.
• The Rust Community page lists IRC channels, forums, and other places where Rust assistance can be obtained.
• Programming Rust: Fast, Safe Systems Development, by Jim Blandy & Jason Orendorff, is a detailed guide to the language.
• rust-learning is a huge collection of assorted Rust resources.

Rust in Firefox

• Developer Documentation
• Build System Documentation
• Rust Update Policy for Firefox
• How to build an XPCOM component in Rust
• In-tree helper crates for Rust XPCOM components
• The #servo IRC channel contains lots of people who know about both Rust and Gecko.
• Are you new to Rust and not sure if your Rust code could be improved? The following people can review Rust patches for Firefox from an "is this good Rust code?" point of view.
  • Alexis Beingessner (:gankro)
  • Josh Bowman-Matthews (:jdm)
  • Emilio Cobos Alvarez (:emilio)
  • Manish Goregaokar (:manishearth)
  • Nika Layzell (:mystor)
  • Cameron McCormack (:heycam)

FAQ

Q: What is the policy for vendoring non-Mozilla crates into mozilla-central?

A: It is possible. The most important point is that the license must be compatible. Reviewers should also look at the crate code some to check that it looks reasonable (especially for unsafe code) and that it has reasonable tests. Other than that, there is no formal sign-off procedure, but one may be added in the future.

Q: Do we support building standalone Rust programs?

A: Yes! Look for RUST_PROGRAMS rules in moz.build files.

Q: How are in-tree Rust crates tested?

A: In general we don't run tests for third-party crates; the assumption is that these crates are sufficiently well-tested elsewhere. (Which means that large test fixtures should be removed when vendoring third-party crates, because they just bloat mozilla-central.) Mozilla crates can be tested with cargo test by adding them to RUST_TESTS in toolkit/library/rust/moz.build. Alternatively, you can write a GTest that uses FFI to call into Rust code.

Rust Components

Within Firefox

Shipped

• MP4 metadata parser: bug 1161350 (shipped for desktop in Firefox 48)
  • Why Rust? Parses untrusted input, replaces libstagefright, a 3rd-party library with a history of security vulnerabilities.
• Replace uconv with encoding-rs: bug 1261841 (shipped in Firefox 56)
• CSS style calculation (from Servo): bug stylo (shipped for desktop in Firefox 57)
  • Why Rust? Code taken from Servo, uses parallel algorithms.
• U2F HID backend: bug 1388843 (shipped in Firefox 57)
• XPIDL binding generator (bug 1293362) (shipped in Firefox 60)
• New prefs parser: bug 1423840 (shipped in Firefox 60)
  • Why Rust? Old parser needed replacing. Well-separated component, simple interface, parses untrusted input.
• Audio remoting for Linux: bug 1434156 (shipped in Firefox 60)
• WebRender: bug webrender (shipped in Firefox 67, enabled for users with appropriate hardware)
  • Why Rust? Code taken from Servo, has high performance; Rust's memory and thread safety provides protection against complexity.
• kvstore (key-value storage backed by LMDB): bug 1490496 (shipped in Firefox 67)
  • Why Rust? The rkv crate provides a safe, ergonomic wrapper around LMDB, our choice for simple key-value storage in Firefox. kvstore wraps rkv in an asynchronous XPCOM API for JS and C++ callers.
• XUL store, backed by rkv: bug 1460811 (landed in Firefox 68, used in Nightly only)
• TLS certificate store, backed by rkv: bug 1429796 (shipped in Firefox 68)
• Synced bookmark merger: bug 1482608 (shipped in Firefox 68, on by default in Nightly and early Beta)
  • Why Rust? Code sharing! The bookmark merging algorithm was factored out into a separate crate, and is shared between Desktop and all our mobile products.
• Windows BITS interface: bug 1520321 (shipped in Firefox 68)
• Japanese encoding detector: bug 1543077 (shipped in Firefox 69)
  • Why Rust? Builds upon encoding_rs, has tiny FFI surface, subject matter prone to accesses past the bounds of a buffer.

In progress

• Integrate fluent-rs, a localization system: bug 1560038
• neqo A QUIC implentation.
• cranelift, a low-level retargetable code generator: bug 1469027
  • Why Rust? It's a new, well-separated component with a clear interface. Also, Rust is a great language for writing compilers, due to algebraic data types and pattern matching.
• Audio remoting for Windows: bug 1432303
• Audio remoting for Mac OS: bug 1425788
• SDP parsing in WebRTC: bug 1365792
  • Why Rust? SDP is a complex text protocol and the existing parser in C has a history of security issues.
• Linebreaking with xi-unicode: bug 1290022 (last update late 2016)
• Background Update Agent for Windows: bug 1343669
• Encoding detector: bug 1551276
  • Why Rust? Builds upon encoding_rs, has tiny FFI surface, subject matter prone to accesses past the bounds of a buffer.

Proposed

• Parallel layout
  • Why Rust? Existing code from Servo, parallel performance.
• Replace the XML parser
  • Why Rust? Parses untrusted input, replaces expat, a 3rd-party library with a history of frequent security vulnerabilities.
• WebMIDI: bug 1201593, bug 1201596, bug 1201598
• Gamepad code: bug 1286699
• Replace the telemetry module(?)
  • Why Rust? The existing C++ code has a history of threading problems.
• Replace DOM serializers (XML, HTML for Save As.., plain text)
  • Why Rust? Need a rewrite anyway. Minor history of security vulnerabilities.
• Image decoders?
  • Why Rust? Parsing untrusted input, some history of security vulnerabilities.
• Expose Rust API to JS Debugger: bug 1263317
• Generate Rust bindings for IPDL actors (bug 1379739)
• WebM demuxer: bug 1267492
• Parallel JS parsing: fast preparse to find function boundaries, parse non-overlapping functions in parallel with a unification step to handle free names and such (no bug on file yet)
  • Why Rust? Parses untrusted input. Requires safe threading. And generally, Rust is a better language than C++ for parsers, due to strong typing, algebraic data types, and pattern matching.
• Crash reporter
  • Why Rust? Code needs rewriting, useful Rust crates exist that could be used.

Outside Firefox

Completed

• Testing
  • GeckoDriver, a WebDriver implementation for Firefox integrated via marionette protocol: bug 1340637 (standalone releases)
  • grcov, a tool to collect and aggregate code coverage data for multiple source files, used in Firefox CI.
• Build system, etc.
  • sccache, compiler cache with s3 storage. Caching C++ and Rust compilation, used in Firefox CI.
  • Parts of mozsearch, the backend for the Searchfox code indexing tool.
  • makecab, a reimplementation of Microsoft's makecab tool. Used to compress PDB files before uploading to symbol server in Firefox CI.
• Application Services, server-side
  • autopush-rs Rust async based websocket server that implements Mozilla's push/webpush/broadcast protocols.
    • Why Rust? Concise code with the memory efficiency of C.
  • Megaphone, a real-time update broadcast server for Firefox.
  • fxa_email_service, a service for sending email to Firefox Accounts.
  • pairsona, a tool to associate instances of firefox.
• Application Services, client-side
  • fxa-rust-client, a cross-compiled FxA Rust client that can work with Firefox Sync keys and more.

In Progress

• IPDL Parser: bug 1316754
  • Why Rust? Rust is a much better language than Python for writing compilers, due to strong typing, algebraic data types, and pattern matching.

Blockers and obstacles

This section lists areas where Rust integration could be improved.

• Tracking bug: Make the developer experience for Firefox + Rust great: bug rust-great
• Compile speed and memory usage
  • Incremental compilation (A-incr-comp issues, WG-compiler-incr issues)
  • Distributed compilation cache
  • Artifact caching?
• Debugging: improve gdb and lldb support for Rust. The first step is to establish Rust language support in DWARF distinct from the existing C++ support.
• Bindings/interop
  • Immature rust-bindgen and cbindgen tools for general cross-language support. Working aroudn clang bugs in different versions and on different platforms can be tricky.
  • No IPDL binding generator (bug 1379739)
  • No WebIDL binding generator for DOM components (Servo must have something here?)
• Remaining minor crash report issues bug 1348896
• IDE/symbol lookup support?
• Code coverage?
• Profiling improvements? Especially for parallel code
• Test integration?

Meetings

• Whistler, Jun 2019
• Orlando, Dec 2018
• San Francisco, Jun 2018
• Austin, Dec 2017
• Mozlando, Dec 2015
• Oxidation, Nov 2015
• Whistler, June 2015
• Mozlandia, Dec 2014