Confirmed users
1,927
edits
Engineering Workflow/Road Map: Difference between revisions
Commit series work
(Auth0) |
(Commit series work) |
||
| (11 intermediate revisions by the same user not shown) | |||
| Line 5: | Line 5: | ||
== Current == | == Current == | ||
=== Phabricator | === Better support for commit series in Phabricator and Lando === | ||
The official command-line interface for Phabricator, Arcanist, does not meet the needs of engineering when it comes to first-class support for submitting, updating, and pulling down series of commits. Lando also does not currently support the atomic landing of an entire series of commits. We will build a new command-line interface to Phabricator and extend Lando to better support standard Firefox engineering practices. | |||
Impact: | Impact: | ||
* | * Efficiently submit and update commit series, encouraging smaller commits. | ||
* Smoothly land commit series, avoiding repetitive clicking and the possibility of merge conflicts. | |||
* | * Eventually, ease the contribution pipeline by not requiring the installation of Arcanist, which can be annoying on some platforms (primarily Windows). | ||
* | |||
=== | === BMO Auth0 Integration === | ||
Our Bugzilla instance will be tied into Mozilla's Auth0 IAM system, which will allow the linking of BMO accounts to LDAP, Mozillians, etc. | |||
Impact: | Impact: | ||
* | * Easily identify employees, NDAed volunteers, and other contributor groups. | ||
* | * Applications integrated with BMO, e.g. Phabricator, can access users' LDAP accounts and other Mozillian data. | ||
* | * Applications already integrated into Mozilla's IAM system can access Bugzilla user data. | ||
=== Vendor manifests === | === Vendor manifests === | ||
| Line 49: | Line 46: | ||
Impact: | Impact: | ||
* Multi-region redundancy and autoscaling for hgweb servers for additional resiliency | * Multi-region redundancy and autoscaling for hgweb servers for additional resiliency. | ||
* Faster checkouts to enable shorter end to end times for local and CI builds | * Faster checkouts to enable shorter end to end times for local and CI builds. | ||
=== Build backend === | === Build backend === | ||
| Line 56: | Line 53: | ||
Implement a new build backend for both local and CI builds. Deploy a new distributed sccache system written in Rust to enable fast, secure local builds in offices. Implement local build telemetry that submits data to the generic data ingestion pipeline to allow us to identify and reduce inefficiencies in local builds. | Implement a new build backend for both local and CI builds. Deploy a new distributed sccache system written in Rust to enable fast, secure local builds in offices. Implement local build telemetry that submits data to the generic data ingestion pipeline to allow us to identify and reduce inefficiencies in local builds. | ||
Impact | Impact: | ||
* Reduce the incidence of clobber builds | * Reduce the incidence of clobber builds. | ||
* Faster incremental builds | * Faster incremental builds. | ||
* Secure and IT supported mechanism for faster local builds in offices | * Secure and IT supported mechanism for faster local builds in offices. | ||
== | == Next == | ||
=== Transplant internal improvements === | |||
The Transplant service is the engine that takes diffs from Phabricator (via Lando) and, until its retirement, MozReview and lands them in the destination repository on behalf of a user. There is some design debt in its architecture and deployability that will benefit future work on related services (particularly Lando). | |||
Impact: | |||
* Lower maintenance costs. | |||
* Easier to develop locally. | |||
* Better support from operations. | |||
=== | === Lando for All === | ||
The initial version of Lando will miss three important features required to be a complete replace for pushing directly to Firefox repositories: support for confidential revisions, and Try, and revision stacks. The latter two were included in MozReview's Autoland system. The former was notably lacking in MozReview. | |||
Impact: | Impact: | ||
* | * Efficiency: users can easily send off a Try job from a Phabricator revision and can easily land multiple revisions at once. | ||
* | * Clarity of tooling and better maintainability, as we can then eliminate patch review in BMO. | ||
* Information clarity: discoverable record of results of Try pushes for a given revision. | |||
== Future == | |||
=== Enforced reviewers === | === Enforced reviewers === | ||
| Line 76: | Line 84: | ||
=== Intent-to-land === | === Intent-to-land === | ||
=== | === Vanilla (cinnabar-less) Git support for mozilla-central === | ||
git-cinnabar was created to allow developers to use Git to interact with Mercurial, mainly [https://hg.mozilla.org|hg.mozilla.org]. It has some downsides, however, such as being a bit tricky to install. Ideally developers would be able to use vanilla Git to contribute to Firefox. | |||
If we view Phabricator and Lando as a conduit to getting code into mozilla-central, then ultimately VCS clients would only need to talk to Phabricator. Phabricator natively supports both Mercurial and Git<sup>[[#vcs-parent-note|1]]</sup>, and full Lando support means that developers will not need to talk to hg.mozilla.org directly. We just need to ensure that the user experience of submitting a patch in progress to Try remains as straightforward as it is today. | |||
Impact: | |||
* No need to maintain nor educate people on git-cinnabar. | |||
* End to unproductive VCS debates. | |||
* Solidifies Phabricator-Lando as the conduit to hg.mozilla.org; this consolidation will make it easier to add more services and automation to the commit pipeline. | |||
= | <div id="vcs-parent-note><small> | ||
1. There is one wrinkle in supporting two VCSes for the same repository: like figuring out parent commits when a different VCS is used to pull down commits; see {{bug|1443375}} for how this was solved with git-cinnabar.</small></div> | |||
=== Lando headless client API support === | === Lando headless client API support === | ||
=== Lando and Phabricator backout support === | === Lando and Phabricator backout support === | ||
=== Run tests from source directory in local and ci builds === | === Run tests from source directory in local and ci builds === | ||
| Line 101: | Line 111: | ||
=== Artifact caching experiments for builds === | === Artifact caching experiments for builds === | ||
== Done == | |||
=== Phabricator integration === | |||
The bulk of Phabricator integration into BMO was completed in May. Phabricator is now available for general usage. | |||
Impact: | |||
* One sole tool for review allows the Engineering Workflow to concentrate its efforts, and eliminates the need for contributors to learn several tools. | |||
* A more intuitive interface than BMO and MozReview, with a contemporary look to appeal to new contributors. | |||
* Some integration with existing tools and processes, balancing complexity and disruption. | |||
* Excellent support from Phabricator's developers ([https://phacility.com Phacility]) and a large user community. | |||
=== Lando initial release === | |||
[https://lando.services.mozilla.com Lando] is the new automatic code-landing tool integrated with Phabricator. It is functionally similar to MozReview's Autoland but is designed to be more maintainable and extensible by being less tightly coupled to the review tool. | |||
Impact: | |||
* One-button landing offers an easier way to land code and minimal context switching from reviewing code. | |||
* Queued landings due to tree closures means developers don't have to be around when they open to land their code. | |||
* Becomes a common gateway to mozilla-central (and other repos) which allows more process and security improvements (see below). | |||