User:Bhearsum/Balrog/Administration
This page describes and documents the web-based administration interface for AUS3.
Overview
The administration interface for AUS3 will be a separate application from the client interface, possibly running on a separate server. It will be the only part of AUS3 with read/write access to the database. Authentication will be handled through LDAP (whether that's done by the web server or by the application is TBD).
Use Cases
Build Machines
Build machines have a relative small set of interactions with the server. The following use cases demonstrate them all:
Releases
At the start of the release process a new release is created in the AUS database. Its entry contains all platform independent information: release name, versions, filenames, bouncer products, etc. No platform information (including a list of platforms to be built) is included in the initial entry. If the release already exists an error will be raised.
At the end of each en-US and l10n repack job the newly created build is added to the AUS database, associated with the release created in the previous step. If the build already exists an error will be raised.
Once all builds for a release have been added to the database the betatest and releasetest channel rules for that stream will be updated to point at the newly created release.
Making the beta & release channels live will not be done by automation, and thus aren't covered here.
Nightly Updates
When a branch that receives nightly updates is set-up, a rule will be added that maps all users on its nightly channel to a release identified by $product-$channel-latest (Eg, Firefox-nightly-latest)1. This is a one time set-up and requires no additional maintenance down the road.
When a new nightly build is ready to be published, the build machine will add it to the AUS database in two places: the $product-$channel-latest release, as well as a $product-$channel-$buildid release. Once the -latest release has been updated the new update is live.
Web Administration
Use Cases
In contrast to build machines, the human-driven interface is capable of modifying all parts of the AUS database. These use cases are highly dependent on UI, so are only bullet points for now:
- Turn off updates for a specific channel, branch, or globally
- Adjust throttling for a specific rule or across a whole branch
- Change an update channel to point at a different release
- Add rules to block a specific architecture
- Add, modify, or delete a custom rule
- Add, modify, or delete a release
UI
TBD
Logging & Rollback
Logging and rollback of will be handled by a shadow tables with two additional fields: username and timestamp. All rows in this table must have a username and timestamp logged. This schema will end up using more space than strictly necessary, but the simplistic nature makes implementation easy. Rules are small enough pieces of data that their shadow table's size should still be negligible. Releases are a big bigger, but we can reduce the size of shadow entries by only storing differences or changes in their JSON blobs, which should be of trivial size. Additionally, it should be a rare case that releases need updating, so their shadow table will not have many rows in it.
When an INSERT to the a table is requested a new row will be inserted into the shadow table containing the rule_id (update_paths) or release name (releases). All other fields (besides username & timestamp) will be NULL. When an UPDATE or DELETE is requested, a new row will be created in the shadow table with identical information to the row being updated.
With the schema being so simplistic there's a lot of onus on the client to present the history in a clear and concise way. I envision a few possible views to make this easy:
- Simple undo ("Revert the last change to the rules")
- Rollback by day/time ("Revert the rules table to the state it was in at noon on August 1st")
- List of recent changes, allowing the user to select the ones they want to revert
- When the details link is clicked, the differences between the current state of the rule and the state before the change is shown. (similar for releases)
[ ] Push 6.0 to the release channel (details) [ ] Push 6.0 to the beta channel (details) [ ] Push 3.6.20 to the beta channel (details) [ ] Push 6.0b5 to the beta channel (details) [ ] Push 6.0b5 to the test channels (details) [ ] Push 3.6.20 to the test channels (details) [SUBMIT]
- Wiki-history-like form that lets you compare/revert to arbitrary states.
The above is by no means a final list or must-have, just some ideas.
Open questions:
- How are updates to shadow table done? Through triggers or by the application?
- Should the application have write access to this table? If so, how do we prevent accidental (or intentional) removal of history?
- How long should we keep shadow table entries for? A month? A year? Forever? This largely depends on whether we care about having long term history or just the ability to roll back quickly.
API
Both the build machine client and the web administration API will make use of the REST interface described below. A 500 response is possible for any method, so not listed for all of them. Error responses (4xx and 5xx) will have an 'errmsg' entity in their response.
| URI | Method | Arguments | Description | Status codes |
|---|---|---|---|---|
| /releases | GET | Retrieves all release names | 200 - Releases returned | |
| 204 - No releases | ||||
| /releases/[name] | GET | Retrieve the release identified by "name" | 200 - Release returned | |
| 404 - Release doesn't exist | ||||
| /releases/[name] | PUT | details - the JSON blob associated with this release | Update the release with the supplied JSON data. If non-existent, it is created. Be careful when using this for existing releases that already have builds added to them -- the JSON blob given here overrides all data for the release, not just platform independent data. | 200 - Release modified |
| 201 - Release created | ||||
| 400 - The details value is malformed or NULL | ||||
| /releases/[name] | DELETE | Delete the release | 200 - Release deleted | |
| 404 - Release does not exist | ||||
| /releases/[name]/builds | GET | Retrieve all builds from a release | 200 - Builds returned | |
| 204 - No builds | ||||
| 404 - Release does not exist | ||||
| /releases/[name]/builds/[platform]/[locale] | GET | Retrieve a specific build from a release | 200 - Build returned | |
| 404 - Release, platform, or locale doesn't exist | ||||
| /releases/[name]/builds/[platform]/[locale] | PUT | details - the JSON blob associated with this build | Update the build with the supplied JSON data. If any of the named release, platform, or locale don't exist, they will be created. If the specific build already exists, it will be overwritten with the supplied information. | 200 - Build modified |
| 201 - Build created | ||||
| 400 - The details value is malformed or NULL. | ||||
| /releases/[name]/builds/[platform]/[locale] | DELETE | Delete the build | 200 - Release deleted | |
| 404 - Build doesn't exist | ||||
| /releases/log | GET | name - Only retrieve log entries for a release with this name | Retrieve history of changes to rules | 200 - Log entries returned |
| product - Only retrieve log entries for a release whose product is this | ||||
| version - Only retrieve log entries for a release whose version is this | 204 - No log entries | |||
| startdate - Only retrieve rules newer than this date/time | ||||
| enddate - Only retrieve rules older than this date/time | ||||
| /rules | POST | priority | Create a new rule with the supplied arguments. | 201 - Rule created |
| mapping | ||||
| throttle | ||||
| update_type | ||||
| product | ||||
| version | ||||
| channel | ||||
| buildTarget | 400 - One or more of the arguments is invalid | |||
| buildID | ||||
| locale | ||||
| osVersion | ||||
| distribution | ||||
| distVersion | ||||
| headerArchitecture | ||||
| comment | ||||
| /rules | GET | Same as /rules POST | Retrieve all of the rules that match the specified arguments. Any missing arguments are regarded as wildcards, therefore a request with no parameters returns all rules. | 200 - Rules returned |
| 204 - No rules | ||||
| /rules/[id] | PUT | Same as /rules POST | Update the rule with the supplied arguments. Any missing arguments will be considered NULL. | 200 - Rule modified |
| 400 - One or more of the arguments is invalid | ||||
| /rules/[id] | POST | Same as /rules POST | Update the rule with any arguments passed. Any missing arguments will not be changed. | 200 - Rule modified |
| 400 - One or more of the arguments is invalid | ||||
| /rules/[id] | DELETE | Delete the rule | 200 - Rule deleted | |
| 404 - Rule does not exist | ||||
| /rules/[id]/rollback | PUT | rules_log_id - Id of an entry in the rules shadow table. Must be an entry with a matching rule id | Rollback the rule to the specified log entry | 200 - Rule modified |
| 400 - rules_log_id doesn't exist or rule_id doesn't match | ||||
| /rules/log | GET | All arguments to /rules POST are valid here | Retrieve history of changes to rules | 200 - Log entries returned |
| startdate - Only retrieve rules newer than this date/time | 204 - No log entries | |||
| enddate - Only retrieve rules older than this date/time |
Footnotes
1. $product-$channel-latest only works in a world where no channel names overlap, which we should be in once 1.9.2 is dead. Doing it this way ensures that we nightly updates work after version bumps happen *and* we don't need to maintain a branch name <-> version mapping a la existing AUS.