User:Bhearsum/Balrog/Administration

This page describes and documents the web-based administration interface for AUS3.

Overview

The administration interface for AUS3 will be a separate application from the client interface, possibly running on a separate server. It will be the only part of AUS3 with read/write access to the database. Authentication will be handled through LDAP (whether that's done by the web server or by the application is TBD).

Use Cases

Build Machines

Build machines have a relative small set of interactions with the server. The following use cases demonstrate them all:

Releases

At the start of the release process a new release is created in the AUS database. Its entry contains all platform independent information: release name, versions, filenames, bouncer products, etc. No platform information (including a list of platforms to be built) is included in the initial entry. If the release already exists an error will be raised.

At the end of each en-US and l10n repack job the newly created build is added to the AUS database, associated with the release created in the previous step. If the build already exists an error will be raised.

Once all builds for a release have been added to the database the betatest and releasetest channel rules for that stream will be updated to point at the newly created release.

Making the beta & release channels live will not be done by automation, and thus aren't covered here.

Nightly Updates

When a branch that receives nightly updates is set-up, a rule will be added that maps all users on its nightly channel to a release identified by $product-$channel-latest (Eg, Firefox-nightly-latest)¹. This is a one time set-up and requires no additional maintenance down the road.

When a new nightly build is ready to be published, the build machine will add it to the AUS database in two places: the $product-$channel-latest release, as well as a $product-$channel-$buildid release. Once the -latest release has been updated the new update is live.

Web Administration

Use Cases

In contrast to build machines, the human-driven interface is capable of modifying all parts of the AUS database. These use cases are highly dependent on UI, so are only bullet points for now:

• Turn off updates for a specific channel, branch, or globally
• Adjust throttling for a specific rule or across a whole branch
• Change an update channel to point at a different release
• Add rules to block a specific architecture
• Add, modify, or delete a custom rule
• Add, modify, or delete a release

UI

TBD

Logging & Rollback

Logging and rollback of will be handled by a shadow tables with two additional fields: username and timestamp. All rows in this table must have a username and timestamp logged. This schema will end up using more space than strictly necessary, but the simplistic nature makes implementation easy. Rules are small enough pieces of data that their shadow table's size should still be negligible. Releases are a big bigger, but we can reduce the size of shadow entries by only storing differences or changes in their JSON blobs, which should be of trivial size. Additionally, it should be a rare case that releases need updating, so their shadow table will not have many rows in it.

When an INSERT to the a table is requested a new row will be inserted into the shadow table containing the rule_id (update_paths) or release name (releases). All other fields (besides username & timestamp) will be NULL. When an UPDATE or DELETE is requested, a new row will be created in the shadow table with identical information to the row being updated.

With the schema being so simplistic there's a lot of onus on the client to present the history in a clear and concise way. I envision a few possible views to make this easy:

• Simple undo ("Revert the last change to the rules")
• Rollback by day/time ("Revert the rules table to the state it was in at noon on August 1st")
• List of recent changes, allowing the user to select the ones they want to revert
  • When the details link is clicked, the differences between the current state of the rule and the state before the change is shown. (similar for releases)

[ ] Push 6.0 to the release channel (details)
[ ] Push 6.0 to the beta channel (details)
[ ] Push 3.6.20 to the beta channel (details)
[ ] Push 6.0b5 to the beta channel (details)
[ ] Push 6.0b5 to the test channels (details)
[ ] Push 3.6.20 to the test channels (details)

[SUBMIT]

• Wiki-history-like form that lets you compare/revert to arbitrary states.

The above is by no means a final list or must-have, just some ideas.

Open questions:

• How are updates to shadow table done? Through triggers or by the application?
• Should the application have write access to this table? If so, how do we prevent accidental (or intentional) removal of history?
• How long should we keep shadow table entries for? A month? A year? Forever? This largely depends on whether we care about having long term history or just the ability to roll back quickly.

API

Both the build machine client and the web administration API will make use of the REST interface described below. A 500 response is possible for any method, so not listed for all of them. Most error responses (4xx and 5xx) will have an 'errmsg' entity in their response. The body of all responses will be a JSON object as described for each resource below.

URI	Method	Arguments	Description	Response status	Response body
/releases	GET		Retrieves all release names	200 - Releases returned	JSON object whose keys are names of releases and values are an object with the following data: product, version and release data
				204 - No releases
/releases/[name]	GET		Retrieve the release identified by "name"	200 - Release returned	JSON object with the following data: product, version, release data
				404 - Release doesn't exist
/releases/[name]	PUT	details - the JSON blob associated with this release	Update the release with the supplied JSON data. If non-existent, it is created. Be careful when using this for existing releases that already have builds added to them -- the JSON blob given here overrides all data for the release, not just platform independent data.	200 - Release modified
				201 - Release created
				400 - The details value is malformed or NULL	errmsg
/releases/[name]	DELETE		Delete the release	200 - Release deleted
				404 - Release does not exist	errmsg
/releases/[name]/builds	GET		Retrieve all builds from a release	200 - Builds returned	JSON object with build data - top-level keys are platforms
				204 - No builds
				404 - Release does not exist	errmsg
/releases/[name]/builds/[platform]/[locale]	GET		Retrieve a specific build from a release	200 - Build returned	JSON object with build data
				404 - Release, platform, or locale doesn't exist	errmsg
/releases/[name]/builds/[platform]/[locale]	PUT	details - the JSON blob associated with this build	Update the build with the supplied JSON data. If any of the named release, platform, or locale don't exist, they will be created. If the specific build already exists, it will be overwritten with the supplied information.	200 - Build modified
				201 - Build created
				400 - The details value is malformed or NULL.	errmsg
/releases/[name]/builds/[platform]/[locale]	DELETE		Delete the build	200 - Release deleted
				404 - Build doesn't exist
/releases/log	GET	name - Only retrieve log entries for a release with this name	Retrieve history of changes to rules	200 - Log entries returned	JSON object whose keys are release_log_id's and values are an object with the following data: release name/version/product/data, username and timestamp
		product - Only retrieve log entries for a release whose product is this
		version - Only retrieve log entries for a release whose version is this		204 - No log entries
		startdate - Only retrieve rules newer than this date/time
		enddate - Only retrieve rules older than this date/time
/rules	POST	priority	Create a new rule with the supplied arguments.	201 - Rule created
		mapping
		throttle
		update_type
		product
		version
		channel
		buildTarget		400 - One or more of the arguments is invalid	errmsg
		buildID
		locale
		osVersion
		distribution
		distVersion
		headerArchitecture
		comment
/rules	GET	Same as /rules POST	Retrieve all of the rules that match the specified arguments. Any missing arguments are regarded as wildcards, therefore a request with no parameters returns all rules.	200 - Rules returned	JSON object whose keys are rule_id's and values are the same as the arguments described above
				204 - No rules
/rules/[id]	PUT	Same as /rules POST	Update the rule with the supplied arguments. Any missing arguments will be considered NULL.	200 - Rule modified
				400 - One or more of the arguments is invalid	errmsg
/rules/[id]	POST	Same as /rules POST	Update the rule with any arguments passed. Any missing arguments will not be changed.	200 - Rule modified
				400 - One or more of the arguments is invalid	errmsg
/rules/[id]	DELETE		Delete the rule	200 - Rule deleted
				404 - Rule does not exist
/rules/[id]/rollback	PUT	rules_log_id - Id of an entry in the rules shadow table. Must be an entry with a matching rule id	Rollback the rule to the specified log entry	200 - Rule modified
				400 - rules_log_id doesn't exist or rule_id doesn't match	errmsg
/rules/log	GET	All arguments to /rules POST are valid here	Retrieve history of changes to rules	200 - Log entries returned	JSON object whose keys are rules_log_id's and values are an object with all rule data, username, and timestamp.
		startdate - Only retrieve rules newer than this date/time		204 - No log entries
		enddate - Only retrieve rules older than this date/time

Footnotes

1. $product-$channel-latest only works in a world where no channel names overlap, which we should be in once 1.9.2 is dead. Doing it this way ensures that we nightly updates work after version bumps happen *and* we don't need to maintain a branch name <-> version mapping a la existing AUS.