Engineering Workflow/Road Map

Engineering Workflow Road Map

These are the improvements and new services we are currently working on and others that we are considering for the near future (1-2 years). It is subject to change.

Current

Phabricator integration

Although our Phabricator instance has been up since September 2017, there has been ongoing work to integrate it into our tools, most notably BMO. There is a tracking bug (bug 1381498) for the remaining work before we announce general availability and start encouraging people to move over.

The last remaining item of note is forking Arcanist so that it works with git-cinnabar (bug 1443375).

Impact:

• One sole tool for review allows the Engineering Workflow to concentrate its efforts, and eliminates the need for contributors to learn several tools.
• A more intuitive interface than BMO and MozReview, with a contemporary look to appeal to new contributors.
• Some integration with existing tools and processes, balancing complexity and disruption.
• Excellent support from Phabricator's developers (Phacility) and a large user community.

Lando initial release

Lando is the new automatic code-landing tool integrated with Phabricator. It is (or at least will be) functionally similar to MozReview's Autoland but is designed to be more maintainable and extensible by being less tightly coupled to the review tool.

Impact:

• One-button landing offers an easier way to land code and minimal context switching from reviewing code.
• Queued landings due to tree closures means developers don't have to be around when they open to land their code.
• Becomes a common gateway to mozilla-central (and other repos) which allows more process and security improvements (see below).

Transplant internal improvements

The Transplant service is the engine that takes diffs from Phabricator (via Lando) and, until its retirement, MozReview and lands them in the destination repository on behalf of a user. There is some design debt in its architecture and deployability that will benefit future work on related services (particularly Lando).

Impact:

• Lower maintenance costs.
• Easier to develop locally.
• Better support from operations.

Vendor manifests

Provide a common structure, process, and tools for managing third-party code in mozilla-central. An Intent to Require was posted to dev.platform on 2018/04/10 and finalized on 2018/10/18. Follow the tracking bug (bug 1454867) for progress.

Impact:

• Easier to identify and find third-party code in mozilla-central.
• Easier to determine metadata about third-party code, including original source and current revision.
• Easier to update third-party code.

BMO usability

Implement a number of long-awaited UX enhancements, including MarkDown, full UTF-8 support, and quick edits.

Impact:

• User delight!
• Less friction for new users.
• Better expressibility and accuracy in comments.

Hg scalability and efficiency

Implement hgweb mirroring in AWS and upstream remotefilelog extension for hg

Impact:

• Multi-region redundancy and autoscaling for hgweb servers for additional resiliency.
• Faster checkouts to enable shorter end to end times for local and CI builds.

Build backend

Implement a new build backend for both local and CI builds. Deploy a new distributed sccache system written in Rust to enable fast, secure local builds in offices. Implement local build telemetry that submits data to the generic data ingestion pipeline to allow us to identify and reduce inefficiencies in local builds.

Impact:

• Reduce the incidence of clobber builds.
• Faster incremental builds.
• Secure and IT supported mechanism for faster local builds in offices.

Next

BMO Auth0 Integration

Our Bugzilla instance will be tied into Mozilla's Auth0 IAM system, which will allow the linking of BMO accounts to LDAP, Mozillians, etc.

Impact:

• Easily identify employees, NDAed volunteers, and other contributor groups.
• Applications integrated with BMO, e.g. Phabricator, can access users' LDAP accounts and other Mozillian data.
• Applications already integrated into Mozilla's IAM system can access Bugzilla user data.

Lando for All

The initial version of Lando will miss three important features required to be a complete replace for pushing directly to Firefox repositories: support for confidential revisions, and Try, and revision stacks. The latter two were included in MozReview's Autoland system. The former was notably lacking in MozReview.

Impact:

• Efficiency: users can easily send off a Try job from a Phabricator revision and can easily land multiple revisions at once.
• Clarity of tooling and better maintainability, as we can then eliminate patch review in BMO.
• Information clarity: discoverable record of results of Try pushes for a given revision.

Future

Vanilla (cinnabar-less) Git support

git-cinnabar was created to allow developers to use Git to interact with Mercurial, mainly [1]. It has some downsides, however, such as being a bit tricky to install. Ideally developers would be able to use vanilla Git to contribute to Firefox.

If we view Phabricator and Lando as a conduit to getting code into mozilla-central, then ultimately VCS clients would only need to talk to Phabricator. Phabricator natively supports both Mercurial and Git¹, and full Lando support means that developers will not need to talk to hg.mozilla.org directly. We just need to ensure that the user experience of submitting a patch in progress to Try remains as straightforward as it is today.

Impact:

• No need to maintain nor educate people on git-cinnabar.
• End to unproductive VCS debates.
• Solidifies Phabricator-Lando as the conduit to hg.mozilla.org; this consolidation will make it easier to add more services and automation to the commit pipeline.

1. There is one wrinkle in supporting two VCSes for the same repository: like figuring out parent commits when a different VCS is used to pull down commits; see bug 1443375 for how this was solved with git-cinnabar.