Sheriffing/How To/Unified Repos: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
m (edit)
(add internal merge tool setup and xcode install as requirements for macOS)
 
(45 intermediate revisions by 6 users not shown)
Line 1: Line 1:
== Unified Repos ==
{{Sheriffing How To|Unified repos}}
I(KWierso)'ll clean this up as I go. Rough IRC logs for now.
= Unified Repos =
 
=== Setting up the repo ===
* http://mozilla-version-control-tools.readthedocs.org/en/latest/hgmozilla/unifiedrepo.html
* http://mozilla-version-control-tools.readthedocs.org/en/latest/hgmozilla/unifiedrepo.html
* https://mozilla-version-control-tools.readthedocs.org/en/latest/hgmozilla/firefoxtree.html
* https://mozilla-version-control-tools.readthedocs.org/en/latest/hgmozilla/firefoxtree.html


Here's how I set it up:
== Setting up the repo ==
* Install Mercurial 3.2 or higher
* Set up your ssh key yourself
* Install [https://www.mercurial-scm.org/downloads Mercurial 4.2] or higher
* Make a copy of your .hgrc file for safekeeping and delete the original
* Make a copy of your .hgrc file for safekeeping and delete the original
* Use default hg tool to resolve code conflicts:
** Open .hgrc file in home folder.
** Search for line <code>[ui]</code>.
** Search for line starting with <code>merge =</code> between the line above and the next one starting with <code>[</code>.
** Add or update it to match <code>merge = internal:merge3</code>
* hg clone https://hg.mozilla.org/mozilla-central
* hg clone https://hg.mozilla.org/mozilla-central
* hg clone https://hg.mozilla.org/hgcustom/version-control-tools
* cd mozilla-central
* cd mozilla-central
* ./mach mercurial-setup
* ./mach mercurial-setup
** Configure mercurial as desired (do not enable the mq extension)
* On macOS
* Edit your global .hgrc file to add the following to the [extensions] section:
** Install xcode which is be required for the execution of some commands: <code>xcode-select --install</code><br>This will open an install dialog which might be minimized. Check the list of open applications and confirm the installation.
** firefoxtree = /path/to/version-control-tools/hgext/firefoxtree
 
* The following will pull all (or at least most) of the branches sheriffs need to have on hand:
Configure mercurial as desired. At a minimum, you need the following:
** hg pull inbound && hg pull b2ginbound && hg pull fx-team && hg pull aurora && hg pull beta && hg pull esr31 && hg pull b2g34 && hg pull b2g32 && hg pull b2g30
* Set your username and email.
** If you need other branches, you can find their names in the [https://hg.mozilla.org/hgcustom/version-control-tools/file/default/pylib/mozautomation/mozautomation/repository.py firefoxtree extension source].
* Enable the firefoxtree extension.
* Enable the histedit extension.
 
The following will pull all (or at least most) of the branches sheriffs need to have on hand:
* hg pull autoland && hg pull beta && hg pull release && hg pull esr52
If you need other branches, you can find their names in the [https://hg.mozilla.org/hgcustom/version-control-tools/file/default/pylib/mozautomation/mozautomation/repository.py firefoxtree extension source].
 
You should now have everything set up for proper use! <br />
 
If you want, you can rename the mozilla-central folder to something like "unified", since it is more than just mozilla-central at this point.


You should now have everything set up for proper use!
The `hg fxheads` command should list references to each of the relevant branch names, and the current revision and commit message for each.
The `hg fxheads` command should list references to each of the relevant branch names, and the current revision and commit message for each.


=== Merges ===
To pull in newly landed changes, you can use one of the following commands:
This assumes you have your unified repo all set up so that `hg fxheads` lists each of central, b2ginbound, fx-team, and inbound.
* hg pull fxtrees # this will pull from all branches listed with the fxheads command
* hg pull treename # this will pull only from the tree with the name "treename"
 
== Merges ==
This assumes you have your unified repo all set up so that `hg fxheads` lists each of central and autoland.


To merge a specific non-tip <revision> from mozilla-inbound to mozilla-central:
To merge a specific non-tip <revision> from autoland to mozilla-central:
# hg pull central
# hg pull central
# hg pull inbound
# hg up central
# hg up central
# hg merge -r <revision>
# hg merge -r <revision>
# hg commit -m "Merge mozilla-inbound to mozilla-central a=merge"
# hg commit -m "Merge autoland to mozilla-central a=merge"
# hg push -r . central
# hg push -r . central


To merge mozilla-central into mozilla-inbound:
To merge mozilla-central into autoland:
# hg pull central
# hg pull central
# hg pull inbound
# hg pull autoland
# hg up inbound
# hg up autoand
# hg merge central
# hg merge central
# hg commit -m "Merge mozilla-central to mozilla-inbound a=merge"
# hg commit -m "Merge mozilla-central to autoland. CLOSED TREE"
# hg push -r . inbound
# hg push -r . autoland
 
* If the `hg merge central` command results in "abort: nothing to merge", you should instead use `hg update -r <mozilla-central's tip revision>` to do a non-merging update, then you can do `hg push -r . autoland` to push it as usual.
 
== Uplifts ==
If branch-specific patches are posted, follow import the patches onto the relevant branch.
 
If you're uplifting directly from an mozilla-central checkin to mozilla-beta:
 
Pull and update to prepare for the uplifts:
* hg pull central
* hg pull aurora
* hg up aurora
Graft the mc commit and bring up the editor to edit the commit message, adding "a=foo" as needed:
* hg graft --edit -r <revision>
Repeat the previous step as needed for all uplifts as needed.
Verify the outgoing changes and push:
* hg out -r . aurora
* hg push -r . aurora
 
You can graft a range of commits at once if that's easier:
* hg graft -r <toprevision>::<bottomrevision>
 
Read the documentation for [https://mozilla-version-control-tools.readthedocs.org/en/latest/hgmozilla/unifiedrepo.html#uplifting-backporting-commits graft] for further help.
 
== Backouts ==
Better tools are coming:
* https://bugzilla.mozilla.org/show_bug.cgi?id=1117632
* https://bugzilla.mozilla.org/show_bug.cgi?id=1121211
<br />
 
The standard backout command makes backing out revisions more difficult than it should, as it won't pre-fill the backout commit message with a bug number:
* Copy backout <revision> from treeherder, note the bug <number>, and remember the <reason>
* hg backout -r <revision>
** This should open up your editor with a completely empty commit message. Type in the backout message like "Backout revision <revision> (bug <number>) for <reason>" (possibly with "CLOSED TREE" in there to get around a closure hook).
* hg push -r . <tree>
 
[https://hg.mozilla.org/hgcustom/version-control-tools/file/4e7ab9638cf0/hgext/qbackout qbackout] can still be used, though you will want to use `hg oops` in place of `hg qbackout` (eg if you use `hg share` to get multiple working directories, qbackout and other uses of mq do not play nice with shared repos):
* Copy the backout <revision> from treeherder
* hg oops -e -r <revision>
** This should open up your editor with a prepopulated commit message like "Backed out <revision> (bug <number>)". Add in the <reason> and possibly "CLOSED TREE" to get around a closure hook.
* hg out -r . <tree>
** to double-check what you will be pushing
* hg push -r . <tree>
 
qbackout can also back out a range of commits in a single backout commit:
* hg oops -e -s -r <toprevision>:<bottomrevision>
** This will open your editor with a prepopulated commit message like "Backed out <toprevision>,<anymiddlerevisions>,<bottomrevision> (bug <number>,any other bug <numbers>)". Add in the <reason> and possibly "CLOSED TREE" to get around a closure hook.
* hg push -r . <tree>
(Omit oops's -s flag to back out each individual revision in the range as a separate commit.)
 
=== Revert a backout ===
* `hg graft`, see https://mozilla-version-control-tools.readthedocs.org/en/latest/hgmozilla/common.html#revive-a-commit-that-was-backed-out
 
== Rebasing after losing a push race ==
This assumes you have commits you're attempting to push to autoland when you lose a push race with someone. Change `autoland` to the correct branch name as needed.
 
* hg pull autoland
* hg rebase -d autoland
* hg push -r . autoland
 
== Recovering from mistakes ==
The histedit subcommand of hg lets you remove, edit and reorder changesets (in addition to other actions):
* hg histedit
If a merge commit needs to be removed:
* hg strip --no-backup -r .
<big>WARNING:</big> The hg strip command below removes any commits that haven't been pushed to the remote repositories, regardless of what branch/label/tag those commits are on. (So if you have local changes as patches for other issues - even based on other trees - they will be removed.) Only do this if you don't care about any of those commits!
* hg strip --no-backup 'not public()'
 
 
You may also need to use some combination of the following to get things back to a known-good state:
* hg purge # http://mercurial.selenic.com/wiki/PurgeExtension
* hg up -C .
 
== See what you're about to push ==
`hg out` without any flags added will show you a LOT of unrelated commits. To see just what you'll be pushing to a given branch, use `hg out -r . <branch>`
* hg out -r . autoland
** This will only display the things you're about to push to the autoland branch


* If the `hg merge central` command results in "abort: nothing to merge", you should instead use `hg update -r <mozilla-central's tip revision>` to do a non-merging update, then you can do `hg push -r . inbound` to push it as usual.
== View incoming changes ==
Since your unified repo will likely have all changesets from the various branches pulled locally, `hg in <somebranch> -r <somerev>` will likely not help you see what would be pulled into your current branch from <somebranch> up to that <somerev>.
Instead, you can use the following to print out the equivalent. Say you want to see what would be merged onto mozilla-central from autoland's revision ca142ec8ba0f:


=== Checkin-neededs ===
Make sure m-c is fully up to date:
=== Uplifts ===
* hg pull central
* hg up central
Get all of autoland's changesets:
* hg pull autoland
Do a preview of the merge without actually performing the merge:
* hg merge -r ca142ec8ba0f -P
This will print out everything on autoland (up to and including revision ca142ec8ba0f) that is not already on mozilla-central.


=== Rebasing after losing a push race ===
== See the log for a particular branch ==
This assumes you have commits you're attempting to push to mozilla-inbound when you lose a push race with someone. Change `inbound` to the correct branch name as needed.
* hg log -fr [branch]


* hg pull inbound
= Extras =
* hg rebase -d inbound
== hg aliases ==
* hg push -r . inbound
=== mergetocentral/mergetointegration ===
I set up the following aliases in my global .hgrc file to make merging things around trunk a little faster:
[alias]
# merge tree $1, revision $2 to central, optionally appending $3 (for "CLOSED TREE", etc)
mergetocentral = !$HG pull central ; $HG up central ; $HG pull $1 ; $HG merge $2 ; $HG commit -m "Merge $1 to central, a=merge $3" ; $HG push -r . central
# merge central tip to tree $1, optionally appending $2 (for "CLOSED TREE", etc)
mergetointegration = !$HG pull central ; $HG pull $1 ; $HG up $1 ; $HG merge central ; $HG commit -m "Merge m-c to $1, a=merge $2" ; $HG push -r . $1


=== Recovering from mistakes ===
With these set up, I can merge autoland's revision 04a3d9130aa0 over to mozilla-central with the following command:
<big>WARNING:</big> This will strip out any commits that haven't been pushed to the remote repositories, regardless of what branch/label/tag those commits are on. (So if you have local changes on inbound and run this command to strip something bad on fx-team, both inbound and fx-team will be stripped.) Only do this if you don't care about any of those commits!
hg mergetocentral autoland 04a3d9130aa0
* hg strip 'not public()'
and then I can merge mozilla-central's tip back to autoland with the following command:
hg mergetointegration autoland
This cuts down on a lot of repetitive typing, saving a bit of time.

Latest revision as of 15:49, 12 July 2022

Unified Repos

Setting up the repo

  • Set up your ssh key yourself
  • Install Mercurial 4.2 or higher
  • Make a copy of your .hgrc file for safekeeping and delete the original
  • Use default hg tool to resolve code conflicts:
    • Open .hgrc file in home folder.
    • Search for line [ui].
    • Search for line starting with merge = between the line above and the next one starting with [.
    • Add or update it to match merge = internal:merge3
  • hg clone https://hg.mozilla.org/mozilla-central
  • cd mozilla-central
  • ./mach mercurial-setup
  • On macOS
    • Install xcode which is be required for the execution of some commands: xcode-select --install
      This will open an install dialog which might be minimized. Check the list of open applications and confirm the installation.

Configure mercurial as desired. At a minimum, you need the following:

  • Set your username and email.
  • Enable the firefoxtree extension.
  • Enable the histedit extension.

The following will pull all (or at least most) of the branches sheriffs need to have on hand:

  • hg pull autoland && hg pull beta && hg pull release && hg pull esr52

If you need other branches, you can find their names in the firefoxtree extension source.

You should now have everything set up for proper use!

If you want, you can rename the mozilla-central folder to something like "unified", since it is more than just mozilla-central at this point.

The `hg fxheads` command should list references to each of the relevant branch names, and the current revision and commit message for each.

To pull in newly landed changes, you can use one of the following commands:

  • hg pull fxtrees # this will pull from all branches listed with the fxheads command
  • hg pull treename # this will pull only from the tree with the name "treename"

Merges

This assumes you have your unified repo all set up so that `hg fxheads` lists each of central and autoland.

To merge a specific non-tip <revision> from autoland to mozilla-central:

  1. hg pull central
  2. hg up central
  3. hg merge -r <revision>
  4. hg commit -m "Merge autoland to mozilla-central a=merge"
  5. hg push -r . central

To merge mozilla-central into autoland:

  1. hg pull central
  2. hg pull autoland
  3. hg up autoand
  4. hg merge central
  5. hg commit -m "Merge mozilla-central to autoland. CLOSED TREE"
  6. hg push -r . autoland
  • If the `hg merge central` command results in "abort: nothing to merge", you should instead use `hg update -r <mozilla-central's tip revision>` to do a non-merging update, then you can do `hg push -r . autoland` to push it as usual.

Uplifts

If branch-specific patches are posted, follow import the patches onto the relevant branch.

If you're uplifting directly from an mozilla-central checkin to mozilla-beta:

Pull and update to prepare for the uplifts:

  • hg pull central
  • hg pull aurora
  • hg up aurora

Graft the mc commit and bring up the editor to edit the commit message, adding "a=foo" as needed:

  • hg graft --edit -r <revision>

Repeat the previous step as needed for all uplifts as needed. Verify the outgoing changes and push:

  • hg out -r . aurora
  • hg push -r . aurora

You can graft a range of commits at once if that's easier:

  • hg graft -r <toprevision>::<bottomrevision>

Read the documentation for graft for further help.

Backouts

Better tools are coming:


The standard backout command makes backing out revisions more difficult than it should, as it won't pre-fill the backout commit message with a bug number:

  • Copy backout <revision> from treeherder, note the bug <number>, and remember the <reason>
  • hg backout -r <revision>
    • This should open up your editor with a completely empty commit message. Type in the backout message like "Backout revision <revision> (bug <number>) for <reason>" (possibly with "CLOSED TREE" in there to get around a closure hook).
  • hg push -r . <tree>

qbackout can still be used, though you will want to use `hg oops` in place of `hg qbackout` (eg if you use `hg share` to get multiple working directories, qbackout and other uses of mq do not play nice with shared repos):

  • Copy the backout <revision> from treeherder
  • hg oops -e -r <revision>
    • This should open up your editor with a prepopulated commit message like "Backed out <revision> (bug <number>)". Add in the <reason> and possibly "CLOSED TREE" to get around a closure hook.
  • hg out -r . <tree>
    • to double-check what you will be pushing
  • hg push -r . <tree>

qbackout can also back out a range of commits in a single backout commit:

  • hg oops -e -s -r <toprevision>:<bottomrevision>
    • This will open your editor with a prepopulated commit message like "Backed out <toprevision>,<anymiddlerevisions>,<bottomrevision> (bug <number>,any other bug <numbers>)". Add in the <reason> and possibly "CLOSED TREE" to get around a closure hook.
  • hg push -r . <tree>

(Omit oops's -s flag to back out each individual revision in the range as a separate commit.)

Revert a backout

Rebasing after losing a push race

This assumes you have commits you're attempting to push to autoland when you lose a push race with someone. Change `autoland` to the correct branch name as needed.

  • hg pull autoland
  • hg rebase -d autoland
  • hg push -r . autoland

Recovering from mistakes

The histedit subcommand of hg lets you remove, edit and reorder changesets (in addition to other actions):

  • hg histedit

If a merge commit needs to be removed:

  • hg strip --no-backup -r .

WARNING: The hg strip command below removes any commits that haven't been pushed to the remote repositories, regardless of what branch/label/tag those commits are on. (So if you have local changes as patches for other issues - even based on other trees - they will be removed.) Only do this if you don't care about any of those commits!

  • hg strip --no-backup 'not public()'


You may also need to use some combination of the following to get things back to a known-good state:

See what you're about to push

`hg out` without any flags added will show you a LOT of unrelated commits. To see just what you'll be pushing to a given branch, use `hg out -r . <branch>`

  • hg out -r . autoland
    • This will only display the things you're about to push to the autoland branch

View incoming changes

Since your unified repo will likely have all changesets from the various branches pulled locally, `hg in <somebranch> -r <somerev>` will likely not help you see what would be pulled into your current branch from <somebranch> up to that <somerev>. Instead, you can use the following to print out the equivalent. Say you want to see what would be merged onto mozilla-central from autoland's revision ca142ec8ba0f:

Make sure m-c is fully up to date:

  • hg pull central
  • hg up central

Get all of autoland's changesets:

  • hg pull autoland

Do a preview of the merge without actually performing the merge:

  • hg merge -r ca142ec8ba0f -P

This will print out everything on autoland (up to and including revision ca142ec8ba0f) that is not already on mozilla-central.

See the log for a particular branch

  • hg log -fr [branch]

Extras

hg aliases

mergetocentral/mergetointegration

I set up the following aliases in my global .hgrc file to make merging things around trunk a little faster:

[alias]
# merge tree $1, revision $2 to central, optionally appending $3 (for "CLOSED TREE", etc)
mergetocentral = !$HG pull central ; $HG up central ; $HG pull $1 ; $HG merge $2 ; $HG commit -m "Merge $1 to central, a=merge $3" ; $HG push -r . central
# merge central tip to tree $1, optionally appending $2 (for "CLOSED TREE", etc)
mergetointegration = !$HG pull central ; $HG pull $1 ; $HG up $1 ; $HG merge central ; $HG commit -m "Merge m-c to $1, a=merge $2" ; $HG push -r . $1

With these set up, I can merge autoland's revision 04a3d9130aa0 over to mozilla-central with the following command:

hg mergetocentral autoland 04a3d9130aa0

and then I can merge mozilla-central's tip back to autoland with the following command:

hg mergetointegration autoland

This cuts down on a lot of repetitive typing, saving a bit of time.