ReleaseEngineering/Applications/ServeS3: Difference between revisions
(Created page with "= What is it? = ServeS3 is a Flask app to serve file listings of an Amazon S3 bucket. We are using it to offload older mozilla-inbound depend builds to S3, then make the archi...") |
(deprecation notice) |
||
| Line 1: | Line 1: | ||
<span style='color: red; font-style: italic; font-size: x-large;'>This application was deprecated in April 2016</span> | |||
= What is it? = | = What is it? = | ||
ServeS3 is a Flask app to serve file listings of an Amazon S3 bucket. We are using it to offload older mozilla-inbound depend builds to S3, then make the archive available to developers & the community in an ftp.m.o like way. It redirects requests to S3, so only supports publicly accessible buckets. | ServeS3 is a Flask app to serve file listings of an Amazon S3 bucket. We are using it to offload older mozilla-inbound depend builds to S3, then make the archive available to developers & the community in an ftp.m.o like way. It redirects requests to S3, so only supports publicly accessible buckets. | ||
Revision as of 09:53, 15 April 2016
This application was deprecated in April 2016
What is it?
ServeS3 is a Flask app to serve file listings of an Amazon S3 bucket. We are using it to offload older mozilla-inbound depend builds to S3, then make the archive available to developers & the community in an ftp.m.o like way. It redirects requests to S3, so only supports publicly accessible buckets.
Source code for the web application lives on hg.m.o (it will move to /build/serveS3 after bug 920892).
Where is it?
The web application is deployed on Amazon's Elastic Beanstalk in the US-West-2 region. The CNAME inbound-archive.pub.build.mozilla.org points at serves3-mozilla-inbound.elasticbeanstalk.com.
The app is deployed through Beanstalk in the AWS Console. Beanstalk also takes care of monitoring, and scaling the number of instances for load.
Requirements/dependencies
The web application depends on some Python packages, which are captured in the requirements.txt file. A virtualenv should be used to install them. The README provides documentation on setup, both production and development instances.
Deployment
Initial Setup
- Get a copy of the source from the repo (see Where is it above)
- copy serveS3-dist.cfg to serveS3.cfg
- fill in the AWS auth info for an account with read only access to your bucket, your bucket and the endpoint (README has some guidance here)
- make sure your bucket is set up with Permissions giving Everyone access to List (ie no support for private buckets)
- create the initial copy of your app
zip `date '+%Y%m%d'`-`hg id -i` makehtml.py requirements.txt application.py serveS3.cfg
- using the Beanstalk part of the AWS console, create a new application:
- Predefined Configuration: Python
- Environment type: Load balancing, autoscaling
- Upload your zip file
- Choose what the <foo> in <foo>.elasticbeanstalk.com should be
- Leave RDS and VPC disabled
- Instance type: the t1.micro default is fine
- Health Check: /
Updating code
- in your copy of the repo, run this command
zip `date '+%Y%m%d'`-`hg id -i` makehtml.py requirements.txt application.py serveS3.cfg
- log in to the AWS console
- click the Upload and Deploy button
- select your new zip file
- wait a few minutes for deployment to happen
Using it
Point your browser at http://inbound-archive.pub.build.mozilla.org/