EngineeringProductivity/HowTo/MirrorRepo: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(→‎Steps: use canonical location of mozilla/master)
 
(13 intermediate revisions by one other user not shown)
Line 1: Line 1:
'''This page is now obselete.  See {{bug|805190}} and https://wiki.mozilla.org/Auto-tools/Projects/MozBase#Mirroring'''
= How to Mirror a Repository in Github to Mozilla-Central =
= How to Mirror a Repository in Github to Mozilla-Central =
== Background ==
== Background ==
Many of our projects (such as [[Auto-tools/Projects/MozBase]] and [https://github.com/mozilla/peptest peptest]) are version controlled with git and github, but are mirrored to mercurial and Mozilla-Central for the purposes of test infrastructure.
Many projects (such as [[Auto-tools/Projects/MozBase]] and [https://github.com/mozilla/peptest peptest]) are version controlled with git and [http://github.com github], but are mirrored to [http://hg.mozilla.org/mozilla-central mozilla-central] for testing infrastructure.


The reasoning is that tools and harnesses such as these are useful (or required) for Mozilla's test infrastructure, but are also useful as standalone utilities. Keeping them in github gives us greater development freedom, makes it easier for new contributors to help out and makes the tools more visible to people outside of the Mozilla community.
The reasoning is that tools and harnesses such as these are useful or required for Mozilla's test infrastructure, but are also useful as standalone utilities. Keeping them in standalone repositories gives us greater development freedom, makes it easier for new contributors to help out and makes the tools more visible to people outside of the Mozilla community.


== Steps ==
== Steps to mirror mozbase ==
This guide uses git [http://learn.github.com/p/tagging.html tags] which are simply a way to assign a name to a commit. This tutorial will use mozbase as an example, though the workflow can apply to any project. Now suppose you fix a critical bug, or add a major new feature that is needed by the test infrastructure in Mozilla-Central (herein called m-c).
This methodology uses git [http://learn.github.com/p/tagging.html tags] which are simply a way to assign a name to a commit. If you fix a critical bug, or add a major new feature that is needed by the test infrastructure in mozilla-central, you'll want to mirror the fix from https://github.com/mozilla/mozbase to http://hg.mozilla.org/mozilla-central (m-c).


# Write your patch for mozbase, attach it to your bug, get it reviewed, check it in to master and resolve the bug fixed [https://wiki.mozilla.org/Auto-tools/Projects/MozBase#Working_on_MozBase_and_contributing_patches as normal]. Now it is time to mirror the latest master to m-c.
# Write your patch for [https://wiki.mozilla.org/Auto-tools/Projects/MozBase mozbase], attach it to your bug, get it reviewed, check it in to master and resolve the bug fixed [https://wiki.mozilla.org/Auto-tools/Projects/MozBase#Working_on_MozBase_and_contributing_patches per the mozbase procedure]. Now it is time to mirror the latest master to m-c.
# Create a new bug called "Mirror mozbase to m-c for xyz" or something like that and add your other bug as a depends on to make it clear why we are mirroring. It is often a good idea to bump the versions on [http://pypi.python.org/pypi pypi] simultaneously with the m-c mirroring.
# Create a new bug called "Mirror mozbase to m-c for xyz" or similar and add your other bug as a dependency to make it clear why mirroring to m-c is desired.
# Copy the entire (up-to-date) mozbase folder from your git repository to the <i>testing/mozbase</i> directory in m-c. Do not include mozdevice, .git, .gitignore, or versionbump.py
# Copy the entire (up-to-date) mozbase folder from https://github.com/mozilla/mozbase to the [https://github.com/mozilla/mozbase testing/mozbase] directory in m-c. Do not include .git, .gitignore, or versionbump.py
# Go to https://github.com/mozilla/mozbase/compare/mozilla-central...HEAD and examine the commits listed there. These are all the commits in master that aren't yet in m-c (i.e this is what you are committing to m-c). Pay special attention for commits that add or remove files and make sure to 'hg add' or 'hg rm' those exact same files in your m-c repository.
# Go to https://github.com/mozilla/mozbase/compare/mozilla-central...HEAD and examine the commits listed there. These are all the commits in master that aren't yet in m-c (i.e this is what you are committing to m-c). Pay special attention for commits that add or remove files and make sure to 'hg add' or 'hg rm' those exact same files in your m-c repository.
# Generate a diff of m-c after you have copied the new mozbase there will files appropriately added and removed. Attach this to the bug.
# Generate a diff of m-c after you have copied the new mozbase there will files appropriately added and removed. Attach this to the bug.
# Test your changes to make sure you didn't break the build using [https://wiki.mozilla.org/ReleaseEngineering/TryServer TryServer].
# Test your changes to make sure you didn't break the build using [https://wiki.mozilla.org/ReleaseEngineering/TryServer TryServer].
# Get the attachment it reviewed, pushed to try and landed on mozilla-inbound. When mozilla-inbound is merged to m-c this bug can be marked fixed.  Make sure your m-c commit message references the mozbase commit hash.
# Get the attachment it reviewed, pushed to try and landed on mozilla-inbound. When mozilla-inbound is merged to m-c this bug can be marked fixed.  '''Make sure your m-c commit message references the mozbase commit hash.'''
# Create a git tag called 'mozilla-central' on your commit, you'll need to delete the old 'mozilla-central' tag. This way, the changeset tagged 'mozilla-central' will always keep track of the current state of mozbase in m-c:
# Create a git tag called 'mozilla-central' on your commit, you'll need to delete the old 'mozilla-central' tag. This way, the changeset tagged 'mozilla-central' will always keep track of the current state of mozbase in m-c.  Use the URL of the mozilla-central changeset hash as the tag commit message:
<pre>
<pre>
git pull --tags git@github.com:mozilla/mozbase.git master
git pull --tags git@github.com:mozilla/mozbase.git master
git tag -d mozilla-central
git tag -d mozilla-central
git tag -a -m "mirrored to m-c" mozilla-central <git changeset of latest commit in m-c>
git tag -a -m "http://hg.mozilla.org/mozilla-central/rev/5ba9fa0f8194" mozilla-central <git changeset of latest commit in m-c>
git push --tags git@github.com:mozilla/mozbase.git master
git push --tags git@github.com:mozilla/mozbase.git master
</pre>
</pre>
Line 25: Line 27:
Note: you can now once again go to https://github.com/mozilla/mozbase/compare/mozilla-central...HEAD to see the state of mozilla-central vs master.
Note: you can now once again go to https://github.com/mozilla/mozbase/compare/mozilla-central...HEAD to see the state of mozilla-central vs master.


A script has been written to aid in this effort: http://k0s.org/mozilla/mozbase/mc-diff.py . It currently hard-codes the path of a pristine mozilla-central clone and does not yet remove hg files removed from git, but other than that it is mostly a general solution to generate the diff
A script is available at [https://mxr.mozilla.org/mozilla-central/source/testing/mozbase/generate_diff.py testing/mozbase/generate_diff.py] to make generating the diff easier.  This script:
 
# Clones the mozbase repository
# Copies the relevant files to [https://mxr.mozilla.org/mozilla-central/source/testing/mozbase/ testing/mozbase]
# Generates a diff appropriate for application to m-c
 
It is '''highly advised''' to use this script versus performing the diff generation versus doing it by hand.
 
See also https://wiki.mozilla.org/Auto-tools/Projects/MozBase#Mirroring for more on mozbase mirroring

Latest revision as of 18:29, 25 February 2016

This page is now obselete. See bug 805190 and https://wiki.mozilla.org/Auto-tools/Projects/MozBase#Mirroring

How to Mirror a Repository in Github to Mozilla-Central

Background

Many projects (such as Auto-tools/Projects/MozBase and peptest) are version controlled with git and github, but are mirrored to mozilla-central for testing infrastructure.

The reasoning is that tools and harnesses such as these are useful or required for Mozilla's test infrastructure, but are also useful as standalone utilities. Keeping them in standalone repositories gives us greater development freedom, makes it easier for new contributors to help out and makes the tools more visible to people outside of the Mozilla community.

Steps to mirror mozbase

This methodology uses git tags which are simply a way to assign a name to a commit. If you fix a critical bug, or add a major new feature that is needed by the test infrastructure in mozilla-central, you'll want to mirror the fix from https://github.com/mozilla/mozbase to http://hg.mozilla.org/mozilla-central (m-c).

  1. Write your patch for mozbase, attach it to your bug, get it reviewed, check it in to master and resolve the bug fixed per the mozbase procedure. Now it is time to mirror the latest master to m-c.
  2. Create a new bug called "Mirror mozbase to m-c for xyz" or similar and add your other bug as a dependency to make it clear why mirroring to m-c is desired.
  3. Copy the entire (up-to-date) mozbase folder from https://github.com/mozilla/mozbase to the testing/mozbase directory in m-c. Do not include .git, .gitignore, or versionbump.py
  4. Go to https://github.com/mozilla/mozbase/compare/mozilla-central...HEAD and examine the commits listed there. These are all the commits in master that aren't yet in m-c (i.e this is what you are committing to m-c). Pay special attention for commits that add or remove files and make sure to 'hg add' or 'hg rm' those exact same files in your m-c repository.
  5. Generate a diff of m-c after you have copied the new mozbase there will files appropriately added and removed. Attach this to the bug.
  6. Test your changes to make sure you didn't break the build using TryServer.
  7. Get the attachment it reviewed, pushed to try and landed on mozilla-inbound. When mozilla-inbound is merged to m-c this bug can be marked fixed. Make sure your m-c commit message references the mozbase commit hash.
  8. Create a git tag called 'mozilla-central' on your commit, you'll need to delete the old 'mozilla-central' tag. This way, the changeset tagged 'mozilla-central' will always keep track of the current state of mozbase in m-c. Use the URL of the mozilla-central changeset hash as the tag commit message:
git pull --tags git@github.com:mozilla/mozbase.git master
git tag -d mozilla-central
git tag -a -m "http://hg.mozilla.org/mozilla-central/rev/5ba9fa0f8194" mozilla-central <git changeset of latest commit in m-c>
git push --tags git@github.com:mozilla/mozbase.git master

Note: you can now once again go to https://github.com/mozilla/mozbase/compare/mozilla-central...HEAD to see the state of mozilla-central vs master.

A script is available at testing/mozbase/generate_diff.py to make generating the diff easier. This script:

  1. Clones the mozbase repository
  2. Copies the relevant files to testing/mozbase
  3. Generates a diff appropriate for application to m-c

It is highly advised to use this script versus performing the diff generation versus doing it by hand.

See also https://wiki.mozilla.org/Auto-tools/Projects/MozBase#Mirroring for more on mozbase mirroring

Retrieved from "https://wiki.allizom.org/index.php?title=EngineeringProductivity/HowTo/MirrorRepo&oldid=1118709"