View source for Thunderbird/Release Driving/Rapid Release Activities/Merge Repositories

== Setting up ==

{{Warning|The merge scripts download the comm-* repository bundles and create clones from those, and update them. These may use a large amount of data bandwidth to complete. This is done to ensure clean repositories.}}

{{Warning|You '''MUST''' disable the firefoxtree tree Mercurial extension before running any of these commands. The names interfere with the repository names used in the scripts.}}

You'll need access to:

* Mercurial (level 3)
* Access to modify [https://treestatus.mozilla.org/ treestatus]
** You can check if you have status by looking for the "assume:project:releng:ci-group:thunderbird-releng" or "assume:project:releng:ci-group:thunderbird-sheriff" permission at https://firefox-ci-tc.services.mozilla.com/profile
** If necessary, the change needs to be made in https://hg.mozilla.org/ci/ci-configuration/file/1d37a3cf95a4e272eeaa7a910193e58ff2028646/grants.yml#l2407. Someone in Firefox Releng will have to land that change.

Clone the drivertools repo

 hg clone http://hg.mozilla.org/users/bugzilla_standard8.plus.com/drivertools/
 cd drivertools/comm-merges

== Merges Overview ==

{{Warning|The comm-release repository is no longer being updated as part of merge day activities as it is no longer used by either Thunderbird or Seamonkey.}}

=== When merges happen ===

There is usually only one merge to perform:

* comm-central -> comm-beta

Once a year, when the first ESR/new release repository is created also run (prior to comm-central -> comm-beta)

* comm-beta -> comm-esrXX

Although the mozilla-beta -> mozilla-release merge is typically done about a week before the mozilla-central -> mozilla-beta merge, the comm merges are done all at once (after the mozilla-central -> mozilla-beta merge). You can see a calendar of when each merge day occurs on the [[Release_Management/Calendar|release calendar]].

On Merge Day, hook up with RelEng (mergeduty) to check when the mozilla-* repositories are being merged. These typically happen sometime in the morning Pacific Time - we aim to do the comm-* repositories '''after''' the mozilla-* ones, so that we don't have broken builds on the branches.

There's also a tool you can use in the drivertools repo that prints out the current mozilla-* and comm-* versions:

  ./get-versions.sh

=== How to close the Trees ===

Use [https://treestatus.mozilla.org/ treestatus] for closing trees. Set the "state" to CLOSED with a reason of "for merges" with the "Planned closure" tick box selected. It is easiest to tick "Remember this change to undo later" since we'll undo these fairly quickly. It is easiest to close these in pairs (e.g. the comm-beta-* together).

Ensure the relevant trees are closed.

* comm-beta
* comm-central

Most of the time it's not necessary to close comm-esrXX trees unless it's that time of year.

=== Reopening the Trees ===

Reopening trees is only done when:

* For central: after the merges are completed and the version numbers have been bumped.
* For beta: This remains closed after a comm-beta -> comm-release merge until after the comm-central -> comm-beta merge has been completed '''and''' the builds have been completed.

When re-opening the tree, ensure you set it to the state that it was previously (generally "approval-required" for beta/esrXX and "open" for central). If "Remember this change to undo later" was ticked while closing the trees you can just click the "Undo" button to set it back to the previous status.

=== Communication about merges ===

The morning of merge day, [mailto:thunderbird-drivers@mozilla.org email the Thunderbird drivers mailing list] with a message reminding them know of the plan for the day. Reply to this email in order to keep people up-to-date with what is happening.

<blockquote>
Hello!

I'll be doing the merges this evening (Eastern time). The mozilla-* merges have <not yet been occurred or already happened>.

comm-central is going to version <new c-c version>, and comm-beta to <new c-b version>.

I'll be following the steps below: (available at [1], this includes updating ship-it/release-services and filing a bug for updating tracking flags). I'll be closing the trees before starting this.

I'll keep people up to date by replying to this email:

* Email before merges begin
* Close the trees
* Perform the merges
* Re-open c-c
* Email after the merges
* Wait for c-b builds to complete
* Set c-b back to approval needed
* Email once the builds are complete

Please let me know if you have any questions or comments.

Thanks,
<your name>

[1] https://wiki.mozilla.org/Thunderbird/Release_Driving/Rapid_Release_Activities/Merge_Repositories
</blockquote>

== Doing the actual merges ==

These assume you've read the previous detail.

The scripts are run from the <code>drivertools/comm-merges</code> directory.

=== comm-beta -> comm-release ===

{{Warning|As of the November 2020 merges, comm-release will no longer be updated.}}

=== comm-beta -> comm-esrXX ===

{{warning|This is only necessary '''ONCE''' per ESR (when a new ESR repository is first created). If you're not sure, you likely want to skip this section.}}

{{Warning|The scripts have not been updated to reflect that comm-release is no longer used. Ideally, automation will be in place before the next ESR is created in July 2021 and it will not be necessary to update them.}}

* Modify the ESR variable in resetesrrepos.sh and merge-release-2-esr.sh to point to the current ESR version.
* Close the relevant trees
* Reset the repos (this removes any previous repository & creates a fresh clone)

  ./resetesrrepos.sh

* Run the script

  ./merge-release-2-esr.sh &> release2esr.log
  less release2esr.log

* Check the log for:
** Any errors
** Outgoing tag to comm-beta
** The diff of changes committed to comm-esr looks OK
** The outgoing changes to comm-esr look ok, including tagging at the start and end of the list.
** Note that the number of outgoing changesets and heads might be large (1000s of commits, 100s of heads) because the comm-esrXXX repository is initially created as a clone of ONLY the default branch of comm-beta. When pushing we'll be pushing all the other release branches, etc.
* Now push the esr changes

  hg -R esr push -f

* comm-esrXXX can now go to approval-needed.

=== comm-central -> comm-beta merge ===

* Close the relevant trees (comm-central-* and comm-beta-*)
* Reset the repos (this removes any previous repository & creates a fresh clone)

  ./resetmainrepos.sh

* Run the script

  ./merge-central-2-beta.sh &> central2beta.log
  less central2beta.log

* Check the log for:
** Any errors
** Outgoing tag to comm-central
** The diff of changes committed to comm-beta looks OK
*** Updating the TaskCluster config to point to the mozilla-beta
** The outgoing changes to comm-beta look ok, including closing of the old head, and tagging at the start and end of the list.
* Now push the central tag (this doesn't cause any builds, as it doesn't change anything)

  hg -R central push

* Check to see if Firefox has tagged a revision for their beta yet. (See below) If not, push to beta. If there is a tagged revision, pin .gecko_rev.yml before pushing to beta.

  hg -R beta push

* For comm-central, the version bumps now need to happen:

  ./bump-versions-central.sh &> bumpcentral.log
  less bumpcentral.log

* Check the log for:
** Any errors
** The diff of changes committed to comm-central looks OK
** The outgoing changes to comm-central look ok, including the changes in version number.
* Now push central changes

  hg -R central push

* comm-central can be reset to it's previous status (generally opened)
* comm-beta doesn't get set back to approval required until after [https://treeherder.mozilla.org/#/jobs?repo=comm-beta builds are completed]

==== Update comm-beta .gecko_rev.yml ====

Check [https://hg.mozilla.org/releases/mozilla-beta/tags] for a tagged revision for beta 1. '''FIREFOX_86_0b1_BUILD2''' or '''FIREFOX_86_0b1_RELEASE'''. If there is no tag yet, the Thunderbird engineer handling the beta release will have to pin later. For now, complete the merge with .gecko_rev.yml pinned to "default".

* Update .gecko_rev.yml with the revision and tag.

  GECKO_BASE_REPOSITORY: https://hg.mozilla.org/mozilla-unified
  GECKO_HEAD_REPOSITORY: https://hg.mozilla.org/releases/mozilla-beta
  GECKO_HEAD_REV: eae58b7bc5654f7cac80985dd647c7558bb88895
  GECKO_HEAD_REF: FIREFOX_77_0b1_BUILD2

* Commit and push that change with CLOSED TREE, e.g.:

  No bug - Pin mozilla-beta (<tag>/<revision>). r+a=me CLOSED TREE

* Go to the comm-beta repository in Treeherder (https://treeherder.mozilla.org/#/jobs?repo=comm-beta).
* Once the build is complete, comm-beta can be set back to approval needed.

== Update Ship It (previously part of release-services, and before that known as ship-it-v1) ==

Mozilla's [https://github.com/mozilla-releng/shipit Ship It] tool must be updated with the updated version number. Ship It is once again its own repository.

As of Thunderbird 83, it looks like the working Shipit development branch has changed from "master" to "main".


=== shipit changes ===

* Modify the [https://github.com/mozilla-releng/shipit/blob/master/api/src/shipit_api/common/config.py <code>api/src/shipit_api/common/config.py</code>] file to update the <code>LATEST_THUNDERBIRD_NIGHTLY_VERSION</code> variable to be the newest version of comm-central.
* Create a pull request with the changes.

An example pull request with the changes (for when comm-central went to 79.0a1) is [https://github.com/mozilla-releng/shipit/pull/237 available to view].

In the pull request, @mention someone from Firefox releng so they see it. You can look at the pull request for updating the Firefox version numbers to see who is working on that release. We have to do this because there's no automatic review on that repository and outside contributors cannot request review themselves.

== Post-Merge tidy ==

Please commit and push any changes to the drivertools repo, for history purposes.