Firefox/Go Faster/Release Pipeline

Proposal

The initial pipeline should be very lightweight. Tests/Builds can be run via a TaskCluster GitHub bridge. The idea being:

• Pull Requests trigger testing (autolanding may need to come later)
• Untagged pushes trigger testing
• Tagged pushes trigger builds which result in an upload being sent to AMO/Balrog

A/B testing and allowing public access to addons will need to be handled by hand after this process is completed.

Questions

platform: Can we get away with doing only Linux initially? There is no good solution for Mac/Windows via TaskCluster at the moment and buildbot changes are not generally within the realm of "light weight."

Pieces Needed

• TaskCluster -> GitHub bridge: this will allow us to set up automated testing and builds very quickly.

• Test/Build TaskCluster Tasks: this can be completed in parallel with the TC-GH bridge.

• TaskCluster Task for Addon uploads: First we need to decide on Balrog or AMO. This task should be a child of a successful build (fetching the built addon from its parent).

Bugs

(mrrrgn) TC->GitHub bridge

Release Triggering

see: ???

Developers will need a system for notifying us when they are ready for a release to be rolled out. We also need procedures for things like tagging.

- Could this be a self-serve UI? - Could this be related to .taskclusterrc and tied to either release tagging or pinning to a revision?

Shipping Infrastructure

See: mrrrgn, JP, ??? (we'll probably need help from other RelEng members)

We could use AMO or Balrog for hosting addons.

AMO Needed Features / Status:

There are some difficulties around using it, but it seems possible to create a working prototype without actually changing its code base (though it will require a workflow with human intervention).

tl;dr: We'll need a special "GoFast" addon type with improved permissions, and a few REST endpoints (instead of crufty old web forms).

Ability to host multiple addon versions at the same time.

Status: This is supported but only for "listed" [public] addons.

Ability to partially roll out updates

Status: The AMO system relies on manifests which map addon versions to supported platforms. Normally users will always get the latest version, but it is possible to package addons with a custom "updateURL" which firefox will use to search for new versions.

We will also need to build our own service for managing version manifests. The service could be a proxy (more flexible) or a web app (easier to manage) depending on what our needs seem to be as the project progresses.

To use updateURL with a listed update, we will need administrator intervention.

API for uploading addons

This is currently a multi-part web form. This will need to be turned into an API asap.

Signing Addons

Signing happens just after an App is uploaded to AMO. Currently addons will be put into a "review queue" with a three month backlog. We can bypass this via an administrator's intervention.