202
edits
Compatibility/System Addon/Release Process: Difference between revisions
Compatibility/System Addon/Release Process (view source)
Revision as of 18:26, 17 May 2022
, 17 May 2022Add documentation for out-of-band updates.
(WIP of an updated version of the Release Process documentation.) |
(Add documentation for out-of-band updates.) |
||
| Line 62: | Line 62: | ||
== Shipping interventions with an out-of-band System Addon Update == | == Shipping interventions with an out-of-band System Addon Update == | ||
'''To be done'''. : | === Prerequisites === | ||
Before you can do any of the work outlined here, you need to make sure that your Mozilla LDAP account is a member of the <tt>vpn_cloudops_shipit</tt> and <tt>xpi_system_build</tt> groups. If you are not sure, [https://firefox-ci-tc.services.mozilla.com/profile check your LDAP profile here (Mozilla-internal)]. If one or both groups are missing, file a bug similar to [https://bugzilla.mozilla.org/show_bug.cgi?id=1683143 bug 1683143 (Mozilla-internal)]. | |||
You also need to be able to connect to the Mozilla Corporate VPN to use ShipIt. See [https://mana.mozilla.org/wiki/display/SD/VPN this guide (Mozilla-internal)] if you're currently not set up for that. | |||
=== Preparations === | |||
Since all changes that are about to be shipped need QA verification, we try to limit the changes to only the things we absolutely need to ship to Release users. As opposed to the other workflows, the primary work for rolling out an update happens on GitHub. Syncing the code back into <tt>mozilla-central</tt> is a step to be done after the update has been kicked off. | |||
The addon's GitHub repository has a <tt>release-hotfix</tt> branch. This branch does not automatically track what's shipping in Firefox Release, and we do not have a process for updating the branch each release cycle. Instead, before doing any work, you have to update the hotfix branch to match the code shipping to Firefox Release. To update that branch: | |||
# Switch your local checkout of the addon sources to the <tt>release-hotfix</tt> branch. Make sure to <tt>git pull</tt>, in case your local checkout is outdated. | |||
# In your <tt>mozilla-central</tt> clone, check out the <tt>mozilla-release</tt> branch/bookmark. | |||
# Check which commit touched the landed addon sources last. If you're using Git for your <tt>mozilla-central</tt>, you can use <tt>git log browser/extensions/webcompat/</tt>. With <tt>git show [the last commit's hash]</tt> check which version number the sources were bumped to. | |||
#* If the version number has a [https://github.com/mozilla-extensions/webcompat-addon/tags corresponding tag in the addon's repository], you can merge that tag into the <tt>release-hotfix</tt> branch with <tt>git merge --ff-only v100.2.1</tt>. | |||
#* If the version number does not have a tag, check [https://github.com/mozilla-extensions/webcompat-addon/commits/main the addon's commit history], and find the commit hash of the backporting commit of that change. Merge the sources up until that commit into the <tt>release-hotfix</tt> branch with <tt>git merge --ff-only [commit hash]</tt>. | |||
#* If the change has not been backported, or you cannot merge a tag/commit cleanly without conflicts, press the "emergency meeting" button in the team. Something has gone really wrong. | |||
If you're done, running the "export into mozilla-central" command should result in no actual changes to your <tt>mozilla-release</tt> checkout. If there are any changes, you merged the wrong tag or commit. Make sure the export is "clean" before proceeding. Otherwise, you might end up accidentally shipping the wrong/additional changes to release. | |||
If the export is not resulting in any changes, <tt>git push</tt> the updated <tt>release-hotfix</tt> branch to GitHub. | |||
Make sure the [[Release Management/Release owners|the Release Manager owning the target Firefox version]] is aware of your intentions to ship an update. Also, have a chat with our Softvision WebCompat team member so they can schedule a bit of time testing your release. | |||
=== Preparing your patch === | |||
# With the addon's <tt>release-hotfix</tt> branch as a base, switch to a new branch for your changes. | |||
# Write and test your interventions as usual. Commit them as usual as well. | |||
# In a separate commit in your branch, bump the version number by a patch-level increase, for example from <tt>100.1.0</tt> to <tt>100.1.1</tt>. Commit that change as well. Make sure that the commit bumping the version number is last in your branch, if it isn't, use <tt>git rebase -i release-hotfix</tt> to reorder the commits. | |||
# Open a PR for your changes. GitHub should figure out that the base branch is <tt>release-hotfix</tt>, but if it doesn't, make sure to set that correctly before submitting the PR. | |||
# Have another member of the team review your PR. | |||
# Test your changes once again, for good measure. Then, merge your PR into the <tt>release-hotfix</tt> branch. | |||
=== Kicking off the build process === | |||
For testing, we do not use the build artifacts for each commit on GitHub, and instead start a new release on ShipIt. The reason we do not use the dep-signed XPI from a commit's job is consistency: When you start a release on ShipIt, it builds an <tt>.xpi</tt> for you to test. In the end, once all testing and QA'ing is done, the same <tt>.xpi</tt> will be shipped to our users, just with a different signature. Essentially, this means we are QA'ing "the same binary" that will be shipped, reducing the chance of weird build-specific glitches. | |||
# Follow the [https://github.com/mozilla-extensions/xpi-manifest/blob/main/docs/releasing-a-xpi.md#starting-the-build-phase "Starting the build phase" steps here]. | |||
#* The list of available XPIs has two WebCompat entires: <tt>webcompat</tt> and <tt>webcompat-release-hotfix</tt>. Pick <tt>webcompat-release-hotfix</tt>, as the other one will build and release the <tt>main</tt> branch. | |||
#* When selecting the revision, double-check that the last revision is correct. This should be a merge commit merging your PR. | |||
# The "Build" link in ShipIt links to taskcluster. You'll see a <tt>dep-signing-webcompat-release-hotfix</tt> task. When the task is completed, the artifacts for that task includes a <tt>webcompat.xpi</tt>. Copy the URL to that. | |||
# Ask our QA friends to test that dep-signed XPI. | |||
#* A template for a QA request can be found [https://bugzilla.mozilla.org/show_bug.cgi?id=1767811#c5 here]. | |||
#* The link to the XPI is the one you copied earlier. | |||
#* Links to current unbranded builds can be found [[Add-ons/Extension_Signing#Unbranded_Builds|here]]. | |||
# After QA is done testing the dep-signed XPI, you need two System Addon admins (or one SAO admin and a Release Engineer as backup) to sign off. Use the <tt>#addons-pipeline</tt> channel on Slack to ask. To make everyone's work easy, provide a link to the bug and a link to the changes (for example a link to the merged GitHub PR). | |||
# When you have received the necessary signoffs, the Ship graph will be triggered, which will sign the .xpi with a Release certificate. Ask a Release Engineer (if in doubt, use the same Slack channel) to get the release-signed .xpi into Balrog's <tt>release-sysaddon</tt> channel. They'll need to know which Firefox version(s) you're targeting with your update, which usually is all minor versions of a given release, i.e. <tt>100.*</tt>. | |||
# When a release engineer created and deployed the <tt>release-sysaddon</tt> rules, ask our QA people to run a second round of testing. | |||
#* A template with all relevant explanations can be found [https://bugzilla.mozilla.org/show_bug.cgi?id=1767811#c9 here]. | |||
#* Note that this should be tested with the real Firefox Release version, not an unbranded build. This testing round is your final chance to catch any errors. | |||
# When your second QA run passed, ask Release Management for formal approval. [https://bugzilla.mozilla.org/show_bug.cgi?id=1767811#c15 This bug comment] is an example. Provide as much detail as you can to make sure Release Management knows everything they need to know. | |||
# After Release Management has signed off, ask a Release Engineer to deploy the rule to the <tt>release</tt> channel. | |||
The update is now live. | |||
=== Follow-up work === | |||
* In the addon's GitHub repo, tag the commit you're rolling out. Push the tag, and merge the <tt>release-hotfix</tt> branch into <tt>main</tt>. This merge will most likely conflict, as the version numbers will disagree. Resolve that conflict by using the current version number from <tt>main</tt> and bump that number by one minor release. | |||
* Even though your code is now shipping to release, please make sure to land your changes to <tt>mozilla-central</tt>, <tt>mozilla-beta</tt>, and <tt>mozilla-release</tt>. Nightly and Beta users will not receive your update, and even for Release users, it's important to ship the new addon version in a dot release, since new profiles otherwise won't get the new interventions immediately and have to wait for the background update. See the uplifting docs above, everything still applies. | |||
* Make sure to label/close all relevant Bugzilla bugs, apply the <tt>sitepatch-applied</tt> label on GitHub, and all the other cleanup work you do in a regular release. | |||
* Optional: You might be interested in keeping an eye on Telemetry data for the rollout. You can create a copy of [https://sql.telemetry.mozilla.org/queries/85734#212283 this telemetry dashboard (Mozilla-internal)], making sure to change the start date, the Firefox version, and the addon version as needed. If you are running into issues, ask :denschub, or ask in <tt>#data-help</tt> on Slack. | |||
= Interventions Release Rotations = | = Interventions Release Rotations = | ||