Phabricator/UpgradeProcess

This document is now moved to Mana

Process

We use two Phabricator clusters: dev and prod. Our dev instance is used essentially as a staging server, that is, releases are deployed to dev for QA to verify before being deployed to prod.

1. Approximately every two weeks, the team picks someone to handle the next upgrade. This role should rotate among the team. We'll call this person the "upgrade mechanic". File a tracking bug filed under Conduit :: Phabricator with the summary "Upgrade Phabricator to week {WEEK}", where {WEEK} is taken from the Phabricator Changelog, e.g. "2018 Week 6".

2. The upgrade mechanic should bump the Phabricator revision hash in a local clone and build the container.

2.1 Update the shas in `mozphab/Dockerfile` as described: https://github.com/darkwing/mozphab/blob/master/Dockerfile#L11

2.2. In the mozphab clone, build the mozphab:test image with: `docker build -t mozilla/mozphab:test .`

2.3. In my phabricator-extensions clone create the phabext:test image

• Modify `Dockerfile`, change the image to `mozilla/mozphab:test`

• Run: `docker build -t mozilla/phabext:test .`

2.4. In my demo clone, change `docker-compose.yml` file's `phabricator->image` to `mozilla/phabext:test`

• make sure you do not override this in `docker-compose.override.yml`

2.5. Run `docker-compose up --build` in the `demo` directory. This will spin up all containers with an additional one called shell with arc installed.

2.6. Make sure that the you configure your Firefox proxy to use the tinyproxy started on docker-ip:1080.

2.7. Go to http://phabricator.test in firefox - you should get an error message about a MySQL upgrade if this upgrade cycle had any.

2.8 Upgrade database: Shell into demo phabricator (`docker exec -it demo_phabricator_1 /bin/sh`), run db migration: `phabricator/ $ ./bin/storage upgrade`

3. The upgrade mechanic performs a simple smoke test: start up the phabricator.test container and log in, post a revision, accept the revision, verify that the attachment is created and r+ed in BMO.

• To run arc commands the upgrade mechanic can use the shell container: `docker-compose run demo`. There is a `version-control-tools` repo configured to run with demo phabricator.

4. If the smoke test fails, the mechanic files bugs as appropriate. At the next standup, determine the priority of these fixes and if they should fit into the current sprint, or if the upgrade should be delayed to the next sprint.

5. If the tests pass immediately, or if the bugs are fixed and the tests then pass, commit the mozphab with updated hash, create the pull request.

6. When the change is merged the new mozilla/mozphab image is built by circle ci and the new docker sha will be at https://hub.docker.com/r/mozilla/mozphab/tags/. Add the new hash to phabricator-extension's Dockerfile, commit and create the revision in Phabricator.

7. After landing the revision file a bug under Cloud Services :: Operations: Phabricator, blocking the tracking bug filed in step 1 above. The description should follow this template:

   Summary: Please deploy Phabricator image {HASH} to dev
   
   Description:
   Phabricator Extension Container Hash: {HASH}
   
   {extra instructions related to this deploy, including addition of new ENV vars}

{HASH} should be replaced by the new hash of the phabext container, which will be at https://hub.docker.com/r/mozilla/phabext/tags/. Also, check the Changelog for database migrations. If the "Migration" section is not empty, indicate that a database migration is necessary in the "extra instructions" field of the deployment bug template.

8. On the following Tuesday, operations will deploy to dev. The upgrade mechanic should do the same simple smoke test on the dev system.

9. The upgrade mechanic should set a needinfo flag to the QA person (e.g. chartjes) on the tracking bug filed in step 1 indicating that the dev deployment is ready to test.

10. QA runs the bigger test plan against dev. Results should be posted in the tracking bug, clearing the needinfo.

11. If the tests passed, the mechanic should file a prod deployment bug, as in step 7 but with "prod" in the summary. Otherwise go to step 4.

12. When prod is deployed, resolve the tracking bug as FIXED.