Sheriffing/How To/Unified Repos: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(Revert backout)
(add internal merge tool setup and xcode install as requirements for macOS)
 
(31 intermediate revisions by 6 users not shown)
Line 1: Line 1:
== Unified Repos ==
{{Sheriffing How To|Unified repos}}
= Unified Repos =
* http://mozilla-version-control-tools.readthedocs.org/en/latest/hgmozilla/unifiedrepo.html
* http://mozilla-version-control-tools.readthedocs.org/en/latest/hgmozilla/unifiedrepo.html
* https://mozilla-version-control-tools.readthedocs.org/en/latest/hgmozilla/firefoxtree.html
* https://mozilla-version-control-tools.readthedocs.org/en/latest/hgmozilla/firefoxtree.html


=== Setting up the repo ===
== Setting up the repo ==
Set up your ssh key yourself
* Set up your ssh key yourself
Install Mercurial 3.2 or higher
* Install [https://www.mercurial-scm.org/downloads Mercurial 4.2] or higher
Make a copy of your .hgrc file for safekeeping and delete the original
* Make a copy of your .hgrc file for safekeeping and delete the original
* Use default hg tool to resolve code conflicts:
** Open .hgrc file in home folder.
** Search for line <code>[ui]</code>.
** Search for line starting with <code>merge =</code> between the line above and the next one starting with <code>[</code>.
** Add or update it to match <code>merge = internal:merge3</code>
* hg clone https://hg.mozilla.org/mozilla-central
* hg clone https://hg.mozilla.org/mozilla-central
* hg clone https://hg.mozilla.org/hgcustom/version-control-tools
* cd mozilla-central
* cd mozilla-central
* ./mach mercurial-setup
* ./mach mercurial-setup
Configure mercurial as desired (do not enable the mq extension)<br />
* On macOS
Edit your global .hgrc file to add the following to the [extensions] section:
** Install xcode which is be required for the execution of some commands: <code>xcode-select --install</code><br>This will open an install dialog which might be minimized. Check the list of open applications and confirm the installation.
*firefoxtree = /path/to/version-control-tools/hgext/firefoxtree
 
Configure mercurial as desired. At a minimum, you need the following:
* Set your username and email.
* Enable the firefoxtree extension.
* Enable the histedit extension.
 
The following will pull all (or at least most) of the branches sheriffs need to have on hand:
The following will pull all (or at least most) of the branches sheriffs need to have on hand:
* hg pull inbound && hg pull b2ginbound && hg pull fx-team && hg pull aurora && hg pull beta && hg pull esr31 && hg pull b2g34 && hg pull b2g32 && hg pull b2g30
* hg pull autoland && hg pull beta && hg pull release && hg pull esr52
If you need other branches, you can find their names in the [https://hg.mozilla.org/hgcustom/version-control-tools/file/default/pylib/mozautomation/mozautomation/repository.py firefoxtree extension source].
If you need other branches, you can find their names in the [https://hg.mozilla.org/hgcustom/version-control-tools/file/default/pylib/mozautomation/mozautomation/repository.py firefoxtree extension source].


You should now have everything set up for proper use! <br />
You should now have everything set up for proper use! <br />


Maybe rename the folder from "mozilla-central" to "unifiedrepo" or something to make it clear it's more than just mozilla-central?
If you want, you can rename the mozilla-central folder to something like "unified", since it is more than just mozilla-central at this point.


The `hg fxheads` command should list references to each of the relevant branch names, and the current revision and commit message for each.
The `hg fxheads` command should list references to each of the relevant branch names, and the current revision and commit message for each.


With [https://bugzilla.mozilla.org/show_bug.cgi?id=1116861 bug 1116861] landed in version-control-tools, you can now use the following commands to pull from groups of repositories with a single command:
To pull in newly landed changes, you can use one of the following commands:
* hg pull fxtrees # this will pull from all branches listed with the fxheads command
* hg pull fxtrees # this will pull from all branches listed with the fxheads command
* hg pull integration # this will pull from the mozilla-inbound, b2g-inbound, and fx-team branches
* hg pull treename # this will pull only from the tree with the name "treename"
* hg pull releases # this will pull from pretty much every release branch (FYI: including old, unused branches)


=== Merges ===
== Merges ==
This assumes you have your unified repo all set up so that `hg fxheads` lists each of central, b2ginbound, fx-team, and inbound.
This assumes you have your unified repo all set up so that `hg fxheads` lists each of central and autoland.


To merge a specific non-tip <revision> from mozilla-inbound to mozilla-central:
To merge a specific non-tip <revision> from autoland to mozilla-central:
# hg pull central
# hg pull central
# hg pull inbound
# hg up central
# hg up central
# hg merge -r <revision>
# hg merge -r <revision>
# hg commit -m "Merge mozilla-inbound to mozilla-central a=merge"
# hg commit -m "Merge autoland to mozilla-central a=merge"
# hg push -r . central
# hg push -r . central


To merge mozilla-central into mozilla-inbound:
To merge mozilla-central into autoland:
# hg pull central
# hg pull central
# hg pull inbound
# hg pull autoland
# hg up inbound
# hg up autoand
# hg merge central
# hg merge central
# hg commit -m "Merge mozilla-central to mozilla-inbound a=merge"
# hg commit -m "Merge mozilla-central to autoland. CLOSED TREE"
# hg push -r . inbound
# hg push -r . autoland


* If the `hg merge central` command results in "abort: nothing to merge", you should instead use `hg update -r <mozilla-central's tip revision>` to do a non-merging update, then you can do `hg push -r . inbound` to push it as usual.
* If the `hg merge central` command results in "abort: nothing to merge", you should instead use `hg update -r <mozilla-central's tip revision>` to do a non-merging update, then you can do `hg push -r . autoland` to push it as usual.


=== Checkin-neededs ===
== Uplifts ==
This assumes we're checking things in to fx-team. Replace fx-team with another branch name as needed.
If branch-specific patches are posted, follow import the patches onto the relevant branch.


Update fx-team and prepare for the patches:
If you're uplifting directly from an mozilla-central checkin to mozilla-beta:
* hg pull fx-team
* hg up fx-team
* hg bookmark fx-team-checkins
Import the patches:
* hg import https://link.to/correct/checkin-needed.patch
* Repeat ^ as needed for all fx-team checkin-needed patches
Verify your outgoing changes:
* hg out -r . fx-team
If you haven't lost a push race:
* hg push -r . fx-team
If you lost a push race:
* hg pull fx-team
* hg rebase -d fx-team
* hg push -r . fx-team
Delete your no longer needed bookmark:
* hg bookmark -d fx-team-checkins
 
=== Uplifts ===
If branch-specific patches are posted, follow the checkin-needed instructions above, importing the patches onto the release branch.
 
If you're uplifting directly from an m-c checkin to aurora:


Pull and update to prepare for the uplifts:
Pull and update to prepare for the uplifts:
Line 88: Line 75:


You can graft a range of commits at once if that's easier:
You can graft a range of commits at once if that's easier:
* hg graft -r <toprevision><bottomrevision>
* hg graft -r <toprevision>::<bottomrevision>


Read the documentation for [https://mozilla-version-control-tools.readthedocs.org/en/latest/hgmozilla/unifiedrepo.html#uplifting-backporting-commits graft] for further help.
Read the documentation for [https://mozilla-version-control-tools.readthedocs.org/en/latest/hgmozilla/unifiedrepo.html#uplifting-backporting-commits graft] for further help.


=== Backouts ===
== Backouts ==
Better tools are coming:
Better tools are coming:
* https://bugzilla.mozilla.org/show_bug.cgi?id=1117632
* https://bugzilla.mozilla.org/show_bug.cgi?id=1117632
Line 100: Line 87:
The standard backout command makes backing out revisions more difficult than it should, as it won't pre-fill the backout commit message with a bug number:
The standard backout command makes backing out revisions more difficult than it should, as it won't pre-fill the backout commit message with a bug number:
* Copy backout <revision> from treeherder, note the bug <number>, and remember the <reason>
* Copy backout <revision> from treeherder, note the bug <number>, and remember the <reason>
* `hg backout -r <revision>`
* hg backout -r <revision>
** This should open up your editor with a completely empty commit message. Type in the backout message like "Backout revision <revision> (bug <number>) for <reason>" (possibly with "CLOSED TREE" in there to get around a closure hook).
** This should open up your editor with a completely empty commit message. Type in the backout message like "Backout revision <revision> (bug <number>) for <reason>" (possibly with "CLOSED TREE" in there to get around a closure hook).
* `hg push -r . <tree>`
* hg push -r . <tree>


[https://hg.mozilla.org/hgcustom/version-control-tools/file/4e7ab9638cf0/hgext/qbackout qbackout] can probably still be used (assuming you don't use `hg share` to get multiple working directories, as mq doesn't play nice with shared repos):
[https://hg.mozilla.org/hgcustom/version-control-tools/file/4e7ab9638cf0/hgext/qbackout qbackout] can still be used, though you will want to use `hg oops` in place of `hg qbackout` (eg if you use `hg share` to get multiple working directories, qbackout and other uses of mq do not play nice with shared repos):
* Copy the backout <revision> from treeherder
* Copy the backout <revision> from treeherder
* `hg qbackout -e -r <revision>`
* hg oops -e -r <revision>
** This should open up your editor with a pre-formed commit message like "Backed out <revision> (bug <number>)". Add in the <reason> and possibly "CLOSED TREE" to get around a closure hook.
** This should open up your editor with a prepopulated commit message like "Backed out <revision> (bug <number>)". Add in the <reason> and possibly "CLOSED TREE" to get around a closure hook.
* `hg qfin -a && hg push -r . <tree>`
* hg out -r . <tree>
** to double-check what you will be pushing
* hg push -r . <tree>


qbackout can also back out a range of commits in a single backout commit:
qbackout can also back out a range of commits in a single backout commit:
* `hg qbackout -e -s -r <toprevision>:<bottomrevision>`
* hg oops -e -s -r <toprevision>:<bottomrevision>
** This will open your editor with a pre-formed commit message like "Backed out <toprevision>,<anymiddlerevisions>,<bottomrevision> (bug <number>,any other bug <numbers>)". Add in the <reason> and possibly "CLOSED TREE" to get around a closure hook.
** This will open your editor with a prepopulated commit message like "Backed out <toprevision>,<anymiddlerevisions>,<bottomrevision> (bug <number>,any other bug <numbers>)". Add in the <reason> and possibly "CLOSED TREE" to get around a closure hook.
* `hg qfin -a ** hg push -r . <tree>`
* hg push -r . <tree>
(Omit qbackout's -s flag to back out each individual revision in the range as a separate commit.)
(Omit oops's -s flag to back out each individual revision in the range as a separate commit.)


==== Revert a backout ====
=== Revert a backout ===
* https://mozilla-version-control-tools.readthedocs.org/en/latest/hgmozilla/common.html#revive-a-commit-that-was-backed-out
* `hg graft`, see https://mozilla-version-control-tools.readthedocs.org/en/latest/hgmozilla/common.html#revive-a-commit-that-was-backed-out


=== Rebasing after losing a push race ===
== Rebasing after losing a push race ==
This assumes you have commits you're attempting to push to mozilla-inbound when you lose a push race with someone. Change `inbound` to the correct branch name as needed.
This assumes you have commits you're attempting to push to autoland when you lose a push race with someone. Change `autoland` to the correct branch name as needed.


* hg pull inbound
* hg pull autoland
* hg rebase -d inbound
* hg rebase -d autoland
* hg push -r . inbound
* hg push -r . autoland


=== Recovering from mistakes ===
== Recovering from mistakes ==
<big>WARNING:</big> This will strip out any commits that haven't been pushed to the remote repositories, regardless of what branch/label/tag those commits are on. (So if you have local changes on inbound and run this command to strip something bad on fx-team, both inbound and fx-team will be stripped.) Only do this if you don't care about any of those commits!
The histedit subcommand of hg lets you remove, edit and reorder changesets (in addition to other actions):
* hg strip 'not public()'
* hg histedit
If a merge commit needs to be removed:
* hg strip --no-backup -r .
<big>WARNING:</big> The hg strip command below removes any commits that haven't been pushed to the remote repositories, regardless of what branch/label/tag those commits are on. (So if you have local changes as patches for other issues - even based on other trees - they will be removed.) Only do this if you don't care about any of those commits!
* hg strip --no-backup 'not public()'


=== See what you're about to push ===
 
You may also need to use some combination of the following to get things back to a known-good state:
* hg purge # http://mercurial.selenic.com/wiki/PurgeExtension
* hg up -C .
 
== See what you're about to push ==
`hg out` without any flags added will show you a LOT of unrelated commits. To see just what you'll be pushing to a given branch, use `hg out -r . <branch>`
`hg out` without any flags added will show you a LOT of unrelated commits. To see just what you'll be pushing to a given branch, use `hg out -r . <branch>`
* hg out -r . inbound
* hg out -r . autoland
** This will only display the things you're about to push to the inbound branch
** This will only display the things you're about to push to the autoland branch


=== View incoming changes ===
== View incoming changes ==
Since your unified repo will likely have all changesets from the various branches pulled locally, `hg in <somebranch> -r <somerev>` will likely not help you see what would be pulled into your current branch from <somebranch> up to that <somerev>.
Since your unified repo will likely have all changesets from the various branches pulled locally, `hg in <somebranch> -r <somerev>` will likely not help you see what would be pulled into your current branch from <somebranch> up to that <somerev>.
Instead, you can use the following to print out the equivalent. Say you want to see what would be merged onto mozilla-central from b2g-inbound's revision ca142ec8ba0f:
Instead, you can use the following to print out the equivalent. Say you want to see what would be merged onto mozilla-central from autoland's revision ca142ec8ba0f:


Make sure m-c is fully up to date:
Make sure m-c is fully up to date:
* hg pull central  
* hg pull central  
* hg up central  
* hg up central  
Get all of b2g-inbound's changesets:
Get all of autoland's changesets:
* hg pull b2ginbound
* hg pull autoland
Do a preview of the merge without actually performing the merge:
Do a preview of the merge without actually performing the merge:
* hg merge -r ca142ec8ba0f -P
* hg merge -r ca142ec8ba0f -P
This will print out everything on b2g-inbound (up to and including revision ca142ec8ba0f) that is not already on mozilla-central.
This will print out everything on autoland (up to and including revision ca142ec8ba0f) that is not already on mozilla-central.
 
== See the log for a particular branch ==
* hg log -fr [branch]
 
= Extras =
== hg aliases ==
=== mergetocentral/mergetointegration ===
I set up the following aliases in my global .hgrc file to make merging things around trunk a little faster:
[alias]
# merge tree $1, revision $2 to central, optionally appending $3 (for "CLOSED TREE", etc)
mergetocentral = !$HG pull central ; $HG up central ; $HG pull $1 ; $HG merge $2 ; $HG commit -m "Merge $1 to central, a=merge $3" ; $HG push -r . central
# merge central tip to tree $1, optionally appending $2 (for "CLOSED TREE", etc)
mergetointegration = !$HG pull central ; $HG pull $1 ; $HG up $1 ; $HG merge central ; $HG commit -m "Merge m-c to $1, a=merge $2" ; $HG push -r . $1
 
With these set up, I can merge autoland's revision 04a3d9130aa0 over to mozilla-central with the following command:
hg mergetocentral autoland 04a3d9130aa0
and then I can merge mozilla-central's tip back to autoland with the following command:
hg mergetointegration autoland
This cuts down on a lot of repetitive typing, saving a bit of time.

Latest revision as of 15:49, 12 July 2022

Unified Repos

Setting up the repo

  • Set up your ssh key yourself
  • Install Mercurial 4.2 or higher
  • Make a copy of your .hgrc file for safekeeping and delete the original
  • Use default hg tool to resolve code conflicts:
    • Open .hgrc file in home folder.
    • Search for line [ui].
    • Search for line starting with merge = between the line above and the next one starting with [.
    • Add or update it to match merge = internal:merge3
  • hg clone https://hg.mozilla.org/mozilla-central
  • cd mozilla-central
  • ./mach mercurial-setup
  • On macOS
    • Install xcode which is be required for the execution of some commands: xcode-select --install
      This will open an install dialog which might be minimized. Check the list of open applications and confirm the installation.

Configure mercurial as desired. At a minimum, you need the following:

  • Set your username and email.
  • Enable the firefoxtree extension.
  • Enable the histedit extension.

The following will pull all (or at least most) of the branches sheriffs need to have on hand:

  • hg pull autoland && hg pull beta && hg pull release && hg pull esr52

If you need other branches, you can find their names in the firefoxtree extension source.

You should now have everything set up for proper use!

If you want, you can rename the mozilla-central folder to something like "unified", since it is more than just mozilla-central at this point.

The `hg fxheads` command should list references to each of the relevant branch names, and the current revision and commit message for each.

To pull in newly landed changes, you can use one of the following commands:

  • hg pull fxtrees # this will pull from all branches listed with the fxheads command
  • hg pull treename # this will pull only from the tree with the name "treename"

Merges

This assumes you have your unified repo all set up so that `hg fxheads` lists each of central and autoland.

To merge a specific non-tip <revision> from autoland to mozilla-central:

  1. hg pull central
  2. hg up central
  3. hg merge -r <revision>
  4. hg commit -m "Merge autoland to mozilla-central a=merge"
  5. hg push -r . central

To merge mozilla-central into autoland:

  1. hg pull central
  2. hg pull autoland
  3. hg up autoand
  4. hg merge central
  5. hg commit -m "Merge mozilla-central to autoland. CLOSED TREE"
  6. hg push -r . autoland
  • If the `hg merge central` command results in "abort: nothing to merge", you should instead use `hg update -r <mozilla-central's tip revision>` to do a non-merging update, then you can do `hg push -r . autoland` to push it as usual.

Uplifts

If branch-specific patches are posted, follow import the patches onto the relevant branch.

If you're uplifting directly from an mozilla-central checkin to mozilla-beta:

Pull and update to prepare for the uplifts:

  • hg pull central
  • hg pull aurora
  • hg up aurora

Graft the mc commit and bring up the editor to edit the commit message, adding "a=foo" as needed:

  • hg graft --edit -r <revision>

Repeat the previous step as needed for all uplifts as needed. Verify the outgoing changes and push:

  • hg out -r . aurora
  • hg push -r . aurora

You can graft a range of commits at once if that's easier:

  • hg graft -r <toprevision>::<bottomrevision>

Read the documentation for graft for further help.

Backouts

Better tools are coming:


The standard backout command makes backing out revisions more difficult than it should, as it won't pre-fill the backout commit message with a bug number:

  • Copy backout <revision> from treeherder, note the bug <number>, and remember the <reason>
  • hg backout -r <revision>
    • This should open up your editor with a completely empty commit message. Type in the backout message like "Backout revision <revision> (bug <number>) for <reason>" (possibly with "CLOSED TREE" in there to get around a closure hook).
  • hg push -r . <tree>

qbackout can still be used, though you will want to use `hg oops` in place of `hg qbackout` (eg if you use `hg share` to get multiple working directories, qbackout and other uses of mq do not play nice with shared repos):

  • Copy the backout <revision> from treeherder
  • hg oops -e -r <revision>
    • This should open up your editor with a prepopulated commit message like "Backed out <revision> (bug <number>)". Add in the <reason> and possibly "CLOSED TREE" to get around a closure hook.
  • hg out -r . <tree>
    • to double-check what you will be pushing
  • hg push -r . <tree>

qbackout can also back out a range of commits in a single backout commit:

  • hg oops -e -s -r <toprevision>:<bottomrevision>
    • This will open your editor with a prepopulated commit message like "Backed out <toprevision>,<anymiddlerevisions>,<bottomrevision> (bug <number>,any other bug <numbers>)". Add in the <reason> and possibly "CLOSED TREE" to get around a closure hook.
  • hg push -r . <tree>

(Omit oops's -s flag to back out each individual revision in the range as a separate commit.)

Revert a backout

Rebasing after losing a push race

This assumes you have commits you're attempting to push to autoland when you lose a push race with someone. Change `autoland` to the correct branch name as needed.

  • hg pull autoland
  • hg rebase -d autoland
  • hg push -r . autoland

Recovering from mistakes

The histedit subcommand of hg lets you remove, edit and reorder changesets (in addition to other actions):

  • hg histedit

If a merge commit needs to be removed:

  • hg strip --no-backup -r .

WARNING: The hg strip command below removes any commits that haven't been pushed to the remote repositories, regardless of what branch/label/tag those commits are on. (So if you have local changes as patches for other issues - even based on other trees - they will be removed.) Only do this if you don't care about any of those commits!

  • hg strip --no-backup 'not public()'


You may also need to use some combination of the following to get things back to a known-good state:

See what you're about to push

`hg out` without any flags added will show you a LOT of unrelated commits. To see just what you'll be pushing to a given branch, use `hg out -r . <branch>`

  • hg out -r . autoland
    • This will only display the things you're about to push to the autoland branch

View incoming changes

Since your unified repo will likely have all changesets from the various branches pulled locally, `hg in <somebranch> -r <somerev>` will likely not help you see what would be pulled into your current branch from <somebranch> up to that <somerev>. Instead, you can use the following to print out the equivalent. Say you want to see what would be merged onto mozilla-central from autoland's revision ca142ec8ba0f:

Make sure m-c is fully up to date:

  • hg pull central
  • hg up central

Get all of autoland's changesets:

  • hg pull autoland

Do a preview of the merge without actually performing the merge:

  • hg merge -r ca142ec8ba0f -P

This will print out everything on autoland (up to and including revision ca142ec8ba0f) that is not already on mozilla-central.

See the log for a particular branch

  • hg log -fr [branch]

Extras

hg aliases

mergetocentral/mergetointegration

I set up the following aliases in my global .hgrc file to make merging things around trunk a little faster:

[alias]
# merge tree $1, revision $2 to central, optionally appending $3 (for "CLOSED TREE", etc)
mergetocentral = !$HG pull central ; $HG up central ; $HG pull $1 ; $HG merge $2 ; $HG commit -m "Merge $1 to central, a=merge $3" ; $HG push -r . central
# merge central tip to tree $1, optionally appending $2 (for "CLOSED TREE", etc)
mergetointegration = !$HG pull central ; $HG pull $1 ; $HG up $1 ; $HG merge central ; $HG commit -m "Merge m-c to $1, a=merge $2" ; $HG push -r . $1

With these set up, I can merge autoland's revision 04a3d9130aa0 over to mozilla-central with the following command:

hg mergetocentral autoland 04a3d9130aa0

and then I can merge mozilla-central's tip back to autoland with the following command:

hg mergetointegration autoland

This cuts down on a lot of repetitive typing, saving a bit of time.