MozillaWiki

Release Management/Merge Documentation

< Release Management
Revision as of 19:49, 8 January 2013 by Akeybl (talk | contribs) (→‎Analyzing for backouts)

NOTE: This needs to be updated around the necessity for confvars changes bug 691866 (first present in Aurora 19)

Archive of Past Merge Days

Release_Management/Merge_Days

What's missing here

This is a document in process, and will also be replaced by automation at some point. In the meantime it outlines the manual steps required on a 'Merge Day' when Mozilla-Aurora becomes Mozilla-Beta and Mozilla-Central becomes Mozilla-Aurora.

ATM there is nothing in here about:

  • synching all the L10N repos (current doc is here for repos: L10n:Migration, and dashboard process is coming)
    • There have been issues with missing l10n locales on first beta1 builds. In m-b --> m-r migration, RelEng run l10n-sync script to uplift beta locales to release, which ensures that all locales have content in release repo and are migrated correctly. However, this script currently doesn't get used for m-a --> m-b merge day (need to check with Axel on his process here).

Beta->Release Merge

Close the Beta Tree

  • Use https://treestatus.mozilla.org/ to close mozilla-beta. No need to reopen till the next merge from Aurora->Beta (or a beta re-spin is necessary)
  • Please use the "Remember previous state" option, so you can easily restore the previous status messages when you are finished (there will be a restore button at the bottom of the page).
  • In order to change tree statuses, your browserID email needs to be added with sheriff permissions (ping edmorley/philor/catlee or ask in #developers).

Sync mozilla-beta to mozilla-release

Mercurial repositories used for final releases should be synced from mozilla-beta.

Tip of default on mozilla-release should be tagged and closed. mozilla-beta should be tagged and pulled. It creates new tip of default on mozilla-release. See Aurora specific sync mechanics for the details.

L10N repositories uses noop (dummy) merge if needed to simplify localizers life.


To handle this process use the following scripts.

Sync mozilla-beta to mozilla-release

Script available here (beta2release.sh) do not use as it doesn't update confvars.sh

Clobber mozilla-release builders

Go to https://secure.pub.build.mozilla.org/clobberer/?branch=mozilla-release and clobber the mozilla-release builders

Clone, Tag, update

NOTE THAT BELOW VERSION IS THE CURRENT FIREFOX VERSION ON M-R 

# Adjust VERSION variable which stands for the current Firefox version in mozilla-release
VERSION=14.0.1
HG_USER="username <email@mozilla.com>"

hg clone http://hg.mozilla.org/releases/mozilla-release
hg clone http://hg.mozilla.org/releases/mozilla-beta

beta_rev=$(hg -R mozilla-beta id -i -r default)
release_rev=$(hg -R mozilla-release id -i -r default)

RELEASE_BASE_TAG="RELEASE_BASE_`date +%Y%m%d`"
RELEASE_TAG="FIREFOX_RELEASE_$VERSION"

hg -R mozilla-beta tag -r $beta_rev -u "$HG_USER" -m "Added tag $RELEASE_BASE_TAG for changeset $beta_rev. CLOSED TREE a=release DONTBUILD" $RELEASE_BASE_TAG
hg -R mozilla-beta push ssh://hg.mozilla.org/releases/mozilla-beta

hg -R mozilla-release tag -r $release_rev -u "$HG_USER" -m "Added tag $RELEASE_TAG for changeset $release_rev. CLOSED TREE a=release" $RELEASE_TAG
hg -R mozilla-release commit -u "$HG_USER" -m "Closing old head. CLOSED TREE a=release" --close-branch

hg -R mozilla-release pull mozilla-beta
hg -R mozilla-release up -C default

Edit mobile mozconfigs

Note: use 'official' for release mozconfigs, not 'release' 

# edit mozconfigs (for mobile) {release,nightly,l10n-{release,nightly}}:
## currently need to update both android and xul
-ac_add_options --with-branding=mobile/android/branding/beta
+ac_add_options --with-branding=mobile/android/branding/official

Commit Changes

# commit the changes
hg -R mozilla-release commit -u "$HG_USER" -m "Updating configs CLOSED TREE a=release ba=release"

# push back
# hg -R mozilla-release push -f ssh://hg.mozilla.org/releases/mozilla-release

Now go watch results

https://tbpl.mozilla.org/?tree=Mozilla-Release

L10N sync

Script available here (beta2release_l10n.sh)

Note for August 24th beta->release uplift, you'll want to incorporate Axel's merge script to deal with when release locale repos don't have anything committed to them yet.

Also note the merge part of this script does not work properly period, and doesn't push to repo 

#!/bin/bash
set -x
set -e

HG_HOST=hg.mozilla.org
HG_USER=lsblakk@mozilla.com
repo=mozilla-release
release_repo_path=releases/l10n/mozilla-release
beta_repo_path=releases/l10n/mozilla-beta
wd=`pwd`/l10n

mkdir -p $wd
cd $wd

for l in `wget -q -O- http://$HG_HOST/releases/$repo/raw-file/default/browser/locales/shipped-locales | grep -v en-US | awk '{print $1}'`; do
    hg clone http://$HG_HOST/$release_repo_path/$l
    hg clone http://$HG_HOST/$beta_repo_path/$l $l.beta
    release_rev=`hg -R $l id -i -r default`
    beta_rev=`hg -R $l.beta id -i -r default`
    hg -R $l pull $l.beta
    hg -R $l up -C default
    heads=`hg -R $l heads --template '{rev}\n' default|wc -l|tr -d ' '`
    if [ "x$heads" != "x1" ]; then
        hg -R $l up -C -r $beta_rev
        HGMERGE=true hg -R $l merge -r $release_rev
        hg -R $l revert -a -y --no-backup -r $beta_rev
        hg -R $l commit -u $HG_USER -m "Merge from mozilla-beta. CLOSED TREE a=release"
    fi
    hg -R $l diff -r $beta_rev -r default
    hg -R $l push -f ssh://$HG_HOST/$release_repo_path/$l
done

Go Re-Open Beta Tree

  • Announce completion of merge in #release-drivers (ping Standard8)

Preparation for Merge Day

Get all of the Repos

$ mdkir ~/Mozilla/
$ cd ~/Mozilla/
$ hg clone http://hg.mozilla.org/mozilla-central/ mozilla-central
$ hg clone http://hg.mozilla.org/releases/mozilla-aurora/ mozilla-aurora
$ hg clone http://hg.mozilla.org/releases/mozilla-beta/ mozilla-beta

Set each Repo up for Pushing

See https://developer.mozilla.org/en/Mercurial_FAQ#Required_configuration

Set up HG Extensions

Edit ~/.hgrc to include 

[extensions]
transplant=
mq=
rebase=
graphlog=

Close the Trees

DO NOT CLOSE mozilla-central FOR MERGING

  • Use https://treestatus.mozilla.org/ to close the trees.
  • Please use the "Remember previous state" option, so you can easily restore the previous status messages when you are finished (there will be a restore button at the bottom of the page).
  • In order to change tree statuses, your browserID email needs to be added with sheriff permissions (ping edmorley/philor/catlee or ask in #developers).

mozilla-central tag/bump from 129 to 130

Preparation and Tagging

  • Send note to dev-planning and #developers announcing that the merge is about to start
$ cd ~/Mozilla/mozilla-central
$ hg pull
$ hg up
$ hg tag FIREFOX_AURORA_129_BASE

Version Bumps

browser/config/version.txt

-129.0a1
+130.0a1

config/milestone.txt

-129.0a1
+130.0a1

js/src/config/milestone.txt

-129.0a1
+130.0a1

mobile/android/confvars.sh

-MOZ_APP_VERSION=129.0a1
+MOZ_APP_VERSION=130.0a1

mobile/xul/confvars.sh

-MOZ_APP_VERSION=129.0a1
+MOZ_APP_VERSION=130.0a1

b2g/confvars.sh

-MOZ_APP_VERSION=129.0a1
+MOZ_APP_VERSION=130.0a1

xpcom/components/Module.h

-  static const unsigned int kVersion = 129;
+  static const unsigned int kVersion = 130;

services/sync/Makefile.in

-weave_version := 1.CURRENT_VERSION.X
+weave_version := 1.NEXT_VERSION.X

Please note that Sync isn't currently using Gecko's version. Instead, the version is a few ahead. Just increment the middle number in the string by 1.

Android Sync uses the milestone directly; see bug 761311.

Commit Your Changes

# check the diffs to ensure nothing additional has been changed
$ hg diff 
$ hg commit -m "Merging in version bump NO BUG"
$ hg out
$ hg push

or, if somebody lands a change on top of m-c in the meantime, 

$ hg pull
$ hg merge
# check the diffs to ensure nothing additional has been changed
$ hg diff 
$ hg commit -m "Merging in version bump NO BUG"
$ hg out
$ hg push
  • Announce to dev-planning and #developers that the m-c bump is completed

mozilla-aurora 54 to mozilla-beta uplift

Tag mozilla-aurora

  • Coordinate with RelEng (Hal) for git.m.o
cd ~/Mozilla/mozilla-aurora
hg tag -m "Tagging for mozilla-aurora->mozilla-beta uplift CLOSED TREE" FIREFOX_BETA_54_BASE
hg out
# review
hg push

Tagging and closing old beta

$ cd ~/mozilla/mozilla-beta
$ hg tag -m "Tagging end of BETA128 CLOSED TREE" FIREFOX_BETA_128_END
$ hg commit --close-branch -m 'closing old head CLOSED TREE'
$ hg out
$ hg push

Pull from Aurora into Beta

$ hg pull -u -r FIREFOX_BETA_54_BASE http://hg.mozilla.org/releases/mozilla-aurora

Version Bumps

browser/config/version.txt

-54.0a2
+54.0

config/milestone.txt

-54.0a2
+54.0

js/src/config/milestone.txt

-54.0a2
+54.0

mobile/android/confvars.sh

# DO NOT blindly s/a2// as it may affect other lines
-MOZ_APP_VERSION=54.0a2
+MOZ_APP_VERSION=54.0

mobile/xul/confvars.sh

# DO NOT blindly s/a2// as it may affect other lines
-MOZ_APP_VERSION=54.0a2
+MOZ_APP_VERSION=54.0

b2g/confvars.sh

-MOZ_APP_VERSION=54.0a2
+MOZ_APP_VERSION=54.0

Commit Version Bumps

# check the diffs to ensure nothing additional has been changed
$ hg diff 
$ hg commit -m "Merging in version bump NO BUG CLOSED TREE"
$ hg out

Branding Changes

mobile/android/config/mozconfigs/android/debug

-ac_add_options --with-branding=mobile/android/branding/aurora
+ac_add_options --with-branding=mobile/android/branding/beta

mobile/android/config/mozconfigs/android/l10n-nightly

-ac_add_options --with-branding=mobile/android/branding/aurora
+ac_add_options --with-branding=mobile/android/branding/beta

mobile/android/config/mozconfigs/android/nightly

-ac_add_options --with-branding=mobile/android/branding/aurora
+ac_add_options --with-branding=mobile/android/branding/beta

mobile/android/config/mozconfigs/android-armv6/debug

-ac_add_options --with-branding=mobile/android/branding/aurora
+ac_add_options --with-branding=mobile/android/branding/beta

mobile/android/config/mozconfigs/android-armv6/l10n-nightly

-ac_add_options --with-branding=mobile/android/branding/aurora
+ac_add_options --with-branding=mobile/android/branding/beta

mobile/android/config/mozconfigs/android-armv6/nightly

-ac_add_options --with-branding=mobile/android/branding/aurora
+ac_add_options --with-branding=mobile/android/branding/beta

mobile/xul/config/mozconfigs/android/debug

-ac_add_options --with-branding=mobile/xul/branding/aurora
+ac_add_options --with-branding=mobile/xul/branding/beta

mobile/xul/config/mozconfigs/android/nightly

-# Nightlies only since this has a cost in performance
-ac_add_options --enable-js-diagnostics

-ac_add_options --enable-application=mobile
+ac_add_options --enable-application=mobile/xul

-ac_add_options --with-branding=mobile/xul/branding/aurora
+ac_add_options --with-branding=mobile/xul/branding/beta

Commit branding changes

$ hg commit -m "Merging in branding changes NO BUG CLOSED TREE ba=release"
$ hg out

L10n data changes

  • confirm with l10n-drivers whether the uplift of locale files will need to be reverted or not (in order to not pull up locales which are not ready for beta)
  • watch mobile/android/locales/all-locales for changes between channels

Example for Aurora 12 -> Beta:

  • MOBILE: no updated file changes desired (rollback is needed)
  • DESKTOP: files on mozilla-aurora are in expected state for beta:
    • upgrade shipped-locales will occur as part of uplift
    • do not upgrade all-locales
hg revert -r FIREFOX_BETA_128_END browser/locales/all-locales
# if mobile
# hg revert -r FIREFOX_AURORA_12_END mobile/android/locales/*-locales

Commit l10n changes

$ hg commit -m "Merging in l10n changes NO BUG CLOSED TREE ba=release"
$ hg out

Analyzing for backouts

See Release_Management/Merge_Documentation#Analyzing_for_backouts_2

Finishing up

  • Use the Regular Clobbers @ clobberer to clobber all beta builders and then...
# ping Standard8 in #r-d to warn about push
hg push -f
  • Notify RelEng (HAL) that the uplift has completed (FF18-specific)

mozilla-central 129 to mozilla-aurora uplift

Tagging and closing old aurora

$ cd ~/mozilla/mozilla-aurora
$ hg tag -m "Tagging for mozilla-central->mozilla-aurora uplift CLOSED TREE" FIREFOX_AURORA_54_END
$ hg commit --close-branch -m 'closing old head CLOSED TREE'
$ hg out
$ hg push

Pull from m-c into Aurora

$ hg pull -u -r FIREFOX_AURORA_129_BASE http://hg.mozilla.org/mozilla-central

Version Bumps

browser/config/version.txt

-129.0a1
+129.0a2

config/milestone.txt

-129.0a1
+129.0a2

js/src/config/milestone.txt

-129.0a1
+129.0a2

b2g/confvars.sh

-MOZ_APP_VERSION=129.0a1
+MOZ_APP_VERSION=129.0a2

mobile/android/confvars.sh

# DO NOT blindly s/a1/a2 - make sure only this line
-MOZ_APP_VERSION=129.0a1
+MOZ_APP_VERSION=129.0a2

mobile/xul/confvars.sh

# DO NOT blindly s/a1/a2 - make sure only this line
-MOZ_APP_VERSION=129.0a1
+MOZ_APP_VERSION=129.0a2

Commit Version Bumps

# check the diffs to ensure nothing additional has been changed
$ hg diff 
$ hg commit -m "Merging in version bump NO BUG CLOSED TREE"
$ hg out

Branding Changes

mobile/android/config/mozconfigs/android/debug

-ac_add_options --with-branding=mobile/android/branding/nightly
+ac_add_options --with-branding=mobile/android/branding/aurora

mobile/android/config/mozconfigs/android/l10n-nightly

-ac_add_options --with-l10n-base=../../l10n-central
+ac_add_options --with-l10n-base=..

-ac_add_options --with-branding=mobile/android/branding/nightly
+ac_add_options --with-branding=mobile/android/branding/aurora

mobile/android/config/mozconfigs/android/nightly

-ac_add_options --with-branding=mobile/android/branding/nightly
+ac_add_options --with-branding=mobile/android/branding/aurora

mobile/android/config/mozconfigs/android-armv6/debug

-ac_add_options --with-branding=mobile/android/branding/nightly
+ac_add_options --with-branding=mobile/android/branding/aurora

mobile/android/config/mozconfigs/android-armv6/l10n-nightly

-ac_add_options --with-l10n-base=../../l10n-central
+ac_add_options --with-l10n-base=..

-ac_add_options --with-branding=mobile/android/branding/nightly
+ac_add_options --with-branding=mobile/android/branding/aurora

mobile/android/config/mozconfigs/android-armv6/nightly

-ac_add_options --with-branding=mobile/android/branding/nightly
+ac_add_options --with-branding=mobile/android/branding/aurora

mobile/android/config/mozconfigs/android-noion/nightly

-ac_add_options --with-branding=mobile/android/branding/nightly
+ac_add_options --with-branding=mobile/android/branding/aurora

mobile/xul/config/mozconfigs/android/debug

-ac_add_options --with-branding=mobile/xul/branding/nightly
+ac_add_options --with-branding=mobile/xul/branding/aurora

mobile/xul/config/mozconfigs/android/nightly

-ac_add_options --with-branding=mobile/xul/branding/nightly
+ac_add_options --with-branding=mobile/xul/branding/aurora

Commit branding changes

$ hg commit -m "Merging in branding changes NO BUG CLOSED TREE ba=release"
$ hg out

Disable profiling

$ for file in `find browser/config/mozconfigs -type f -name nightly`; do sed -i -e '/enable-profiling/d' $file; done
$ hg diff
$ hg commit -m "Disable profiling on Aurora NO BUG CLOSED TREE ba=release"
$ hg out

L10n data changes

  • confirm with l10n-drivers whether the uplift of locale files will need to be reverted or not (in order to not pull up locales which are not ready for beta)
    • On Aurora, it appears we should do this every time (but we do need to resolve things like bug 785652 – adding zh-TW and zh-CN to fennec (FF19 specific). Beta needs confirmation each release.
  • watch mobile/android/locales/all-locales for changes between channels
hg revert -r FIREFOX_AURORA_54_END browser/locales/*-locales
hg revert -r FIREFOX_AURORA_54_END mobile/android/locales/*-locales

Commit l10n changes

$ hg commit -m "Merging in l10n changes NO BUG CLOSED TREE ba=release"
$ hg out

Analyzing for backouts

Use this report to find all approvals for Aurora/Beta over the past cycle, then plop that buglist into a query like this to find a shortened list of things we may need to take on the next Aurora/Beta version

Known Backouts for Version

For Aurora 19,

  • 803630 - Remove the visible UI setting for enabling reflow on zoom (Fx for Android :mfinkle)

For Beta 19 (and probably 20)

  • 817775 - Marketplace promo is active on beta on about:home, but it shouldn't be (Fx for Android :mfinkle)

For 20?

  • 791244 – Undo the pref for the temporary hangserver for beta channels until Socorro is awesome

Releng to move Aurora to 10.7 builders

When Fedora 14 is merged to mozilla-aurora, release engineering will need to land and reconfig for bug 744445. This will make mozilla-aurora start building for Mac on 10.7. NOTE: this will need to be done for all Fedora 14 merge days. bug 744450 tracks the ff14 beta actions and bug 744456 tracks ff14 release actions for this project. A section like this should be added to the next merge day documentation

Finishing up

  • Use the Regular Clobbers @ clobberer to clobber all aurora builders and then...
# ping Standard8 (or TB driver) in #r-d to warn about push
hg push -f

Completing the Merge

  • Email dev.planning and let them know the merge is complete
  • Notify #developers
  • re-open Aurora, Beta trees
  • Update the merge docs as needed to keep them up to date for next merge day
  • file a bug requesting the disabling of Aurora updates (eg: bug 700794) - until we have sign off from QA
  • clone the RelEng tracking bug (eg: bug 764909) to track the next merge day's custom automation concerns
  • New: remind Jorge that he needs to prevent an add-on compatibility bump for the hotfix add-ons

Change Template Version Numbers

Now that everything has been bumped edit the following so our Channel Notes and other wiki pages have the correct info and queries now (increment by 1)

Retrieved from "https://wiki.allizom.org/index.php?title=Release_Management/Merge_Documentation&oldid=497566"