ReleaseEngineering/Applications/RelengAPI: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
No edit summary
(Shrunk in deference to the README)
Line 1: Line 1:
{{warning|This is still under development!}}
As [[ReleaseEngineering/Applications]] attests, releng has a lot of apps, and many of them have APIs.
As [[ReleaseEngineering/Applications]] attests, releng has a lot of apps, and many of them have APIs.
But these APIs are all bespoke implementations, and are not tied to a single endpoint, authentication scheme, etc.
But these APIs are all bespoke implementations, and are not tied to a single endpoint, authentication scheme, etc.
Line 5: Line 4:


= Goals =
= Goals =
* Simple self-service usage for consumers
* Simple self-service usage for consumers
** Industry-standard access mechanisms (REST, oAuth2, etc.) that require no client-side custom libraries
** One or very few endpoints (e.g., https://api.pub.build.mozilla.org)
** Self-documenting tools
** Semantic versioning
* Simple, rapid implementation of new apps
* Simple, rapid implementation of new apps
** Common requirements such as authentication, database access, scheduled tasks, configuration handling are already satisfied
** All apps use the same technologies (language, web framework, DB framework, etc.), so the learning curve from one app to the next is small
** Tailored for easy local development - minimal requirements, minimal installed components, etc.
* Operations-friendly
* Operations-friendly
** Horizontally scalable using normal webops techniques
See [https://github.com/mozilla/build-relengapi#goals README.md] for more
** Easily deployed in multiple environments with normal devops processes
** Resilient to failure: no in-memory state
 
= Implementation =
 
* Code: https://github.com/djmitche/relengapi
* Implementation: https://api-pub-build.allizom.org (staging)
 
The releng API is composed of a [http://flask.pocoo.org/ Flask] application and a number of [http://flask.pocoo.org/docs/blueprints/ blueprints] installed as separate Python packages.
The base application registers any blueprints installed in the python environment, and provides access to all of the common services.
Development can take place with as few or as many blueprints installed as desired.
 
== Configuration ==
 
Configuration comes from a typical Flask configuration file: a single Python file defining a number of variables.
 
== Databases ==
 
RelengAPI handles accessing multiple databases simultaneously, identified by short names like 'scheduler'.
Each blueprint, as well as the RelengAPI base, can specify and access tables in any database.
Database access information is specified in the configuration.
A command-line subcommand is provided to automatically create all attached tables.
 
Support for Alembic is planned, enabling automatic, graceful upgrades and downgrades of databases.
 
== Authentication ==
 
RelengAPI currently supports BrowserID for user authentication, and oAuth2 for authentication of client applications.
 
== Job Workers ==
 
RelengAPI integrates with Celery to support workers.
 
== Documentation ==
 
General documentation is in Sphinx format and is automatically served by the API itself.
API endpoints, database tables, and so on will be automatically documented as well.
 
== Command Line ==
 
Support is included for adding subcommands to the 'relengapi' command for specific blueprints.
 
== Proxies ==
 
In many cases, especially for transitioning new services into relengapi, we'll want to proxy from relengapi to other HTTP APIs.
 
RelengAPI provides a simple mechanism to configure an API endpoint to proxy to another HTTP endpoint.


= Deployment =
= Links =


The plan is to deploy this on the releng web cluster, a group of VMs running Apache behind a load-balancer in scl3.
* Code: https://github.com/mozilla/build-relengapi
This is simplest in terms of securing access to critical resources like databases and other hosts on the releng network.
* Deployment: https://api.pub.build.mozilla.org (production) or https://api-pub-build.allizom.org (staging)
However, nothing in the design of the system precludes hosting on services like [http://aws.amazon.com/elasticbeanstalk/ Elastic Beanstalk], [https://www.heroku.com/ Heroku], or [[Paas_Apps|PaaS]].

Revision as of 19:19, 4 April 2014

As ReleaseEngineering/Applications attests, releng has a lot of apps, and many of them have APIs. But these APIs are all bespoke implementations, and are not tied to a single endpoint, authentication scheme, etc. RelengAPI aims to be the glue that binds all of these apps together, makes it easy to add new ones, and makes it easy for others to build tools that consume releng's APIs.

Goals

  • Simple self-service usage for consumers
  • Simple, rapid implementation of new apps
  • Operations-friendly

See README.md for more

Links

Retrieved from "https://wiki.allizom.org/index.php?title=ReleaseEngineering/Applications/RelengAPI&oldid=961119"