Oxidation: Difference between revisions

1,148 bytes removed ,  17 November 2020
no edit summary
(→‎In progress: Adding the Glean SDK)
No edit summary
 
(21 intermediate revisions by 5 users not shown)
Line 74: Line 74:
== Rust in Firefox ==
== Rust in Firefox ==


* [https://developer.mozilla.org/en-US/Firefox/Building_Firefox_with_Rust_code Developer Documentation]
Policy
* [https://firefox-source-docs.mozilla.org/build/buildsystem/rust.html Build System Documentation]
* [[Rust_Update_Policy_for_Firefox|Rust Update Policy for Firefox]]
* [[Rust_Update_Policy_for_Firefox|Rust Update Policy for Firefox]]
* [https://docs.google.com/presentation/d/1qkPwISU1BvsTVyqLuVhisSMuIS_DiXb8X09-boszcu0/edit#slide=id.g5bfdbbe5c9_0_71 How to build an XPCOM component in Rust]
 
* [https://groups.google.com/forum/#!topic/mozilla.dev.platform/u8scZop3FkM In-tree helper crates for Rust XPCOM components]
Rust in Firefox docs
* The Rust Matrix channel contains lots of people who know about both Rust and Gecko.
* [https://firefox-source-docs.mozilla.org/build/buildsystem/rust.html Adding Rust code]
* [https://firefox-source-docs.mozilla.org/writing-rust-code/index.html Writing Rust code]
* [https://firefox-source-docs.mozilla.org/testing-rust-code/index.html Testing & debugging Rust code]
 
Getting extra help
* The Rust channel on [https://wiki.mozilla.org/Matrix Mozilla's Matrix network] contains lots of people who know about both Rust and Gecko.
* Are you new to Rust and not sure if your Rust code could be improved? The following people can review Rust patches for Firefox from an "is this good Rust code?" point of view.
* Are you new to Rust and not sure if your Rust code could be improved? The following people can review Rust patches for Firefox from an "is this good Rust code?" point of view.
** Alexis Beingessner (:Gankra)
** Alexis Beingessner (:Gankra)
Line 88: Line 92:
** Cameron McCormack (:heycam)
** Cameron McCormack (:heycam)


== FAQ ==
= Rust Components =


'''Q:''' What is the policy for vendoring non-Mozilla crates into mozilla-central?<br />
== Within Firefox ==
 
'''A:''' It is possible. The most important point is that the license must be compatible. Reviewers should also look at the crate code some to check that it looks reasonable (especially for unsafe code) and that it has reasonable tests. Other than that, there is no formal sign-off procedure, but one may be added in the future.
 
 
'''Q:''' Do we support building standalone Rust programs?<br />
 
'''A:''' Yes! Look for <tt>RUST_PROGRAMS</tt> rules in <tt>moz.build</tt> files.
 
 
'''Q:''' How are in-tree Rust crates tested?<br />


'''A:''' In general we don't run tests for third-party crates; the assumption is that these crates are sufficiently well-tested elsewhere. (Which means that large test fixtures should be removed when vendoring third-party crates, because they just bloat mozilla-central.) Mozilla crates can be tested with <tt>cargo test</tt> by adding them to <tt>RUST_TESTS</tt> in <tt>toolkit/library/rust/moz.build</tt>. Alternatively, you can write a GTest that uses FFI to call into Rust code.
=== Statistics ===


= Rust Components =
* [https://docs.google.com/spreadsheets/d/1flUGg6Ut4bjtyWdyH_9emD9EAN01ljTAVft2S4Dq620/edit#gid=885787479 Lines of compiled Rust code shipped in Firefox (over time)]
 
* [https://4e6.github.io/firefox-lang-stats/ Lines of Rust code in mozilla-central (current)]
== Within Firefox ==


=== Shipped ===
=== Shipped ===
Line 117: Line 110:
* U2F HID backend: {{bug|1388843}} (shipped in Firefox 57)
* U2F HID backend: {{bug|1388843}} (shipped in Firefox 57)
* libcubeb Audio backend for Linux (PulseAudio): {{bug|1346665}} (shipped in Firefox 59)
* libcubeb Audio backend for Linux (PulseAudio): {{bug|1346665}} (shipped in Firefox 59)
* libcubeb Audio backend for macOS (CoreAudio): {{bug|1530713}} (shipped in Firefox 74)
** '''Why Rust?''' Safer implementation avoids data racing by leveraging lifetime, variable mutability check and better mutex pattern.
* XPIDL binding generator ({{bug|1293362}}) (shipped in Firefox 60)
* XPIDL binding generator ({{bug|1293362}}) (shipped in Firefox 60)
* New prefs parser: {{bug|1423840}} (shipped in Firefox 60)
* New prefs parser: {{bug|1423840}} (shipped in Firefox 60)
Line 145: Line 140:
* Encoding detector: {{bug|1551276}} (shipped in Firefox 73)
* Encoding detector: {{bug|1551276}} (shipped in Firefox 73)
** '''Why Rust?''' Builds upon encoding_rs, has tiny FFI surface, subject matter prone to accesses past the bounds of a buffer, potentially parallelizable with Rayon.
** '''Why Rust?''' Builds upon encoding_rs, has tiny FFI surface, subject matter prone to accesses past the bounds of a buffer, potentially parallelizable with Rayon.
* Integrate [https://github.com/projectfluent/fluent-rs fluent-rs], a localization system: {{bug|1560038}} (shipped in Firefox 76)
** '''Why Rust?''' Performance and memory wins are substantial over previous JS implementation. It brings zero-copy parsing, and memory savvy resolving of localization strings. It also paves the way for migrating the rest of the Fluent APIs away from JS which is required for Fission.
* qcms [https://searchfox.org/mozilla-central/source/gfx/qcms] ported from C to Rust
** '''Why Rust?''' Memory safety


=== In progress ===
=== In progress ===


* Integrate [https://github.com/projectfluent/fluent-rs fluent-rs], a localization system: {{bug|1560038}}
* Port [https://github.com/projectfluent/fluent-rs Localization API] to Rust: {{bug|1613705}}
* [https://github.com/mozilla/neqo neqo] A QUIC implentation.
* [https://github.com/mozilla/neqo neqo] A QUIC implentation.
* [https://github.com/CraneStation/cranelift/ cranelift, a low-level retargetable code generator]: {{bug|1469027}}
* [https://github.com/CraneStation/cranelift/ cranelift, a low-level retargetable code generator]: {{bug|1469027}}
** '''Why Rust?''' It's a new, well-separated component with a clear interface. Also, Rust is a great language for writing compilers, due to algebraic data types and pattern matching.
** '''Why Rust?''' It's a new, well-separated component with a clear interface. Also, Rust is a great language for writing compilers, due to algebraic data types and pattern matching.
* [https://github.com/mozilla-spidermonkey/rust-frontend Project Visage] A Rust front-end for SpiderMonkey (featuring [https://github.com/mozilla-spidermonkey/jsparagus jsparagus]).
* [https://github.com/mozilla-spidermonkey/rust-frontend SmooshMonkey] A Rust front-end for SpiderMonkey (featuring [https://github.com/mozilla-spidermonkey/jsparagus jsparagus]).
** '''Why Rust?''' Parses untrusted input. Also, Rust is a great language for writing compilers, due to algebraic data types and pattern matching.
** '''Why Rust?''' Parses untrusted input. Also, Rust is a great language for writing compilers, due to algebraic data types and pattern matching.
* Audio remoting for Windows: {{bug|1432303}}
* Audio remoting for Windows: {{bug|1432303}}
Line 160: Line 159:
* Linebreaking with xi-unicode: {{bug|1290022}} (last update late 2016)
* Linebreaking with xi-unicode: {{bug|1290022}} (last update late 2016)
* Background Update Agent for Windows: {{bug|1343669}}
* Background Update Agent for Windows: {{bug|1343669}}
* libcubeb Audio backend for macOS (CoreAudio): {{bug|1530715}} (in Nightly since 70?)
* [https://github.com/gfx-rs/wgpu wgpu], a [https://gpuweb.github.io/gpuweb/ WebGPU] API implementation: {{bug|webgpu-mvp}} (in Nightly since 72)
* [https://github.com/gfx-rs/wgpu wgpu], a [https://gpuweb.github.io/gpuweb/ WebGPU] API implementation: {{bug|webgpu-mvp}} (in Nightly since 72)
** '''Why Rust?''' Complex tracking logic, wide attack area. Also, leverages Rust ecosystem for building libraries on top of our native implementation and the API that will target the Web.
** '''Why Rust?''' Complex tracking logic, wide attack area. Also, leverages Rust ecosystem for building libraries on top of our native implementation and the API that will target the Web.
Line 187: Line 185:
* Rewrite the ICE stack used by WebRTC; {{bug|1616966}}
* Rewrite the ICE stack used by WebRTC; {{bug|1616966}}
** '''Why Rust?''' Works with network data.
** '''Why Rust?''' Works with network data.
== Outside Firefox ==
== Outside Firefox ==


Line 208: Line 207:
* Application Services, client-side
* Application Services, client-side
** [https://github.com/mozilla/application-services/tree/master/components various sync-related components used on iOS and Fenix], includes a cross-compiled FxA Rust client, and storage/syncing of bookmarks, history, logins, tabs and webextensions data.
** [https://github.com/mozilla/application-services/tree/master/components various sync-related components used on iOS and Fenix], includes a cross-compiled FxA Rust client, and storage/syncing of bookmarks, history, logins, tabs and webextensions data.
* [https://github.com/mozilla/fix-stacks/ fix-stacks], a stack frame symbolizer: {{bug|1596292}}
** '''Why Rust?''' High performance needed, a single implementation can replace multiple platform-specific scripts, and we can use the [https://github.com/getsentry/symbolic/ symbolic] crate to do all the hard parts.


=== In Progress ===
=== In Progress ===
Line 213: Line 214:
* IPDL Parser: {{bug|1316754}}
* IPDL Parser: {{bug|1316754}}
** '''Why Rust?''' Rust is a much better language than Python for writing compilers, due to strong typing, algebraic data types, and pattern matching.
** '''Why Rust?''' Rust is a much better language than Python for writing compilers, due to strong typing, algebraic data types, and pattern matching.
* [https://github.com/mozilla/fix-stacks/ fix-stacks], a stack frame symbolizer: {{bug|1596292}}
** '''Why Rust?''' High performance needed, a single implementation can replace multiple platform-specific scripts, and we can use the [https://github.com/getsentry/symbolic/ symbolic] crate to do all the hard parts.


= Blockers and obstacles =
= Blockers and obstacles =
Line 220: Line 219:
This section lists areas where Rust integration could be improved.
This section lists areas where Rust integration could be improved.
* Tracking bug: Make the developer experience for Firefox + Rust great: {{Bug|rust-great}}
* Tracking bug: Make the developer experience for Firefox + Rust great: {{Bug|rust-great}}
* Compile speed and memory usage
* [https://docs.google.com/document/d/16FgQPRxNb-Z6sfJy_P7edXzMVWE86jJEZVbmV0oC-m0/ 2020 Questionnaire results]
** Incremental compilation ([https://github.com/rust-lang/rust/labels/A-incr-comp A-incr-comp issues], [https://github.com/rust-lang/rust/labels/WG-compiler-incr WG-compiler-incr issues])
* [https://docs.google.com/document/d/1puZvhWaURtViz8OC0HkB0h2dqJThVyAgLGFbIQpd4fo/ Oxidation 2020 Plan]
** [https://users.rust-lang.org/t/contract-opportunity-mozilla-distributed-compilation-cache-written-in-rust/13898 Distributed compilation cache]
** [https://github.com/rust-lang/cargo/issues/1997 Artifact caching]?
* Debugging: improve gdb and lldb support for Rust. The first step is to establish Rust language support in DWARF distinct from the existing C++ support.
* Bindings/interop
** Immature rust-bindgen and cbindgen tools for general cross-language support. Working aroudn clang bugs in different versions and on different platforms can be tricky.
** No IPDL binding generator ({{bug|1379739}})
** No WebIDL binding generator for DOM components (Servo must have something here?)
* Remaining minor crash report issues {{bug|1348896}}
* IDE/symbol lookup support?
* Code coverage?
* Profiling improvements? Especially for parallel code
* Test integration?


= Meetings =
= Meetings =
Confirmed users
523

edits