Release Management/Merge Documentation

NOTE: For the next merge, we need to check for interface changes that are unpaired to UUID changes

NOTE: Release Management is trying out scripts to automate the merge day process, the work on updating the wiki and complete automation is in progress. Hence let us know if you find anything missing here

Archive of Past Merge Days

Release_Management/Merge_Days

What's missing here

This is a document in process, and will also be replaced by automation at some point. In the meantime it outlines the manual steps required on a 'Merge Day' when Mozilla-Aurora becomes Mozilla-Beta and Mozilla-Central becomes Mozilla-Aurora.

ATM there is nothing in here about:

• synching all the L10N repos (current doc is here for repos: L10n:Migration, and dashboard process is coming)
  • There have been issues with missing l10n locales on first beta1 builds. In m-b --> m-r migration, RelEng run l10n-sync script to uplift beta locales to release, which ensures that all locales have content in release repo and are migrated correctly. However, this script currently doesn't get used for m-a --> m-b merge day (need to check with Axel on his process here).
• Thunderbird/SeaMonkey merging, that doc is at Thunderbird/Release_Driving/Rapid_Release_Activities/Merge_Repositories

Mozilla-Beta -> Mozilla-Release Merge

• Make sure that there isn't an l10n locales bug that needs to filed like: bug 826825

Close the Beta Tree

• Use https://treestatus.mozilla.org/ to close mozilla-beta. No need to reopen till the next merge from Aurora->Beta (or a beta re-spin is necessary)
  • Please use the "Remember previous state" option, so you can easily restore the previous status messages when you are finished (there will be a restore button at the bottom of the page).
  • In order to change tree statuses, your browserID email needs to be added with sheriff permissions (ping edmorley/philor/catlee or ask in #developers).

Sync mozilla-beta to mozilla-release

Mercurial repositories used for final releases should be synced from mozilla-beta.

Tip of default on mozilla-release should be tagged and closed. mozilla-beta should be tagged and pulled. It creates new tip of default on mozilla-release. See Aurora specific sync mechanics for the details.

L10N repositories uses noop (dummy) merge if needed to simplify localizers life.

To handle this process use the following scripts.

Beta to Release Merge day helper Script

(Beta2Release) Merge day helper script at: https://github.com/mozilla/relman-tools
ALWAYS GET A FRESH COPY IN CASE THERE HAVE BEEN UPDATES!!!
Use script beta2release.py for this merge day

Clobber mozilla-release builders

Go to https://secure.pub.build.mozilla.org/clobberer/?branch=mozilla-release and clobber the mozilla-release builders

Clone, Tag, update [Taken care by the helper script beta2release.py]

Running beta2release.py

• Save this the script in the directory where you have checked out the repos needed to do the beta to release merge.

• Script will take-care of tagging, return here when ready to push mozilla-beta

# follow instructions in script combined with wiki commands

• • Next step is to run the script : python beta2release.py

hg -R mozilla-beta push ssh://hg.mozilla.org/releases/mozilla-beta

Edit desktop config

# edit mozilla-release/browser/locales/shipped-locales file if you need to remove some beta locales (eg mn, sw)
# script takes care of config changes

Edit mobile mozconfigs

Note: use 'official' for release mozconfigs, not 'release'

# script takes care of config changes

Commit Changes

# commit the changes
#script takes care of the commits

# push back
# hg -R mozilla-release push -f ssh://hg.mozilla.org/releases/mozilla-release

Now go watch results

https://tbpl.mozilla.org/?tree=Mozilla-Release

L10N sync

• Note you need permission to be able to run this script successfully.File a server ops bug similar to bug 869654

Script available here (beta2release_l10n.sh)

Note for August 24th beta->release uplift, you'll want to incorporate Axel's merge script to deal with when release locale repos don't have anything committed to them yet.

Also note the merge part of this script does not work properly period, and doesn't push to repo

#!/bin/bash
set -x
set -e

HG_HOST=hg.mozilla.org
HG_USER=lsblakk@mozilla.com
repo=mozilla-release
release_repo_path=releases/l10n/mozilla-release
beta_repo_path=releases/l10n/mozilla-beta
wd=`pwd`/l10n

mkdir -p $wd
cd $wd

for l in `wget -q -O- http://$HG_HOST/releases/$repo/raw-file/default/browser/locales/shipped-locales | grep -v en-US | awk '{print $1}'`; do
    hg clone http://$HG_HOST/$release_repo_path/$l
    hg clone http://$HG_HOST/$beta_repo_path/$l $l.beta
    release_rev=`hg -R $l id -i -r default`
    beta_rev=`hg -R $l.beta id -i -r default`
    hg -R $l pull $l.beta
    hg -R $l up -C default
    heads=`hg -R $l heads --template '{rev}\n' default|wc -l|tr -d ' '`
    if [ "x$heads" != "x1" ]; then
        hg -R $l up -C -r $beta_rev
        HGMERGE=true hg -R $l merge -r $release_rev
        hg -R $l revert -a -y --no-backup -r $beta_rev
        hg -R $l commit -u $HG_USER -m "Merge from mozilla-beta. CLOSED TREE a=release"
    fi
    hg -R $l diff -r $beta_rev -r default
    hg -R $l push -f ssh://$HG_HOST/$release_repo_path/$l
done

Go Re-Open Beta Tree

• Announce completion of merge in #release-drivers (ping Standard8 and Callek)

Merge Day (Central, Aurora, Beta)

Merge day helper repo : https://github.com/mozilla/relman-tools/blob/master/merge-day-scripts/merge_helper.py ALWAYS GET A FRESH COPY IN CASE THERE HAVE BEEN UPDATES!!!

• Use the merge_helper.py script
• File a bug to disable Aurora updates like bug 931832 but for the version you are moving off Aurora to Beta
• Make sure the trees are green and as much as possible land the aurora approved patches

Check IIDs that need to be bumped

• Find IIDs that need to be bumped from Aurora to Beta: https://github.com/jwir3/checkiid
  • the command is -I (not small 'L' which is what it looks like on the readme)

# run this in an updated mozilla-aurora dir
hg diff -r FIREFOX_AURORA_128_END -r tip -I "**.idl" > /tmp/firefox.diff

• • ie: idl files that changed on Aurora and didn't get an IID bump
  • need to get landed before 54 Beta 1

Get all of the Repos

mkdir ~/Mozilla/
cd ~/Mozilla/
hg clone http://hg.mozilla.org/mozilla-central/ mozilla-central
hg clone http://hg.mozilla.org/releases/mozilla-aurora/ mozilla-aurora
hg clone http://hg.mozilla.org/releases/mozilla-beta/ mozilla-beta

Set each Repo up for Pushing

See https://developer.mozilla.org/en/Mercurial_FAQ#Required_configuration

Set up HG Extensions

Edit ~/.hgrc to include

[extensions]
transplant=
mq=
rebase=
graphlog=

Close the Trees

DO NOT CLOSE mozilla-central FOR MERGING

• Use https://treestatus.mozilla.org/ to close the trees.
• Please use the "Remember previous state" option, so you can easily restore the previous status messages when you are finished (there will be a restore button at the bottom of the page).
• In order to change tree statuses, your browserID email needs to be added with sheriff permissions (ping edmorley/philor/catlee or ask in #developers).

Ping Release Engineering

Release Engineering will need to pause the hg-git sync to gecko.git. If we don't pause that job, and we push the merges to central, aurora, and beta, we will most likely get the next gecko train's code into the wrong b2g versioned branch, and partners will have to deal with a non-fastforward commit.

Get Helper Script (do this every merge day)

# Make sure you get a fresh copy in case there have been updates since last merge
git clone https://github.com/mozilla/relman-tools/merge_day_scripts
use the merge_helper.py script for this merge day

• Open two terminal windows, be in the top dir of your cloned repos in both
• In one of them run the helper script (in the other you can run the manual steps: tagging, pushing, etc):

python merge_helper.py
# follow instructions in script combined with wiki commands

mozilla-central tag/bump from 129 to 130

Preparation and Tagging

• Send note to dev-planning and #developers announcing that the merge is about to start

• Script will do tagging of mozilla_central -> FIREFOX_AURORA_129_BASE
• Script will do version bumps

Verify & Push Your Changes

# check the diffs to ensure nothing additional has been changed
hg push

• Announce to dev-planning and #developers that the m-c bump is completed

mozilla-aurora 54 to mozilla-beta uplift

Review tagging of mozilla-aurora and push

# review
cd ~/Mozilla/mozilla-aurora
hg push

Review tagging of mozilla-beta and continue to push

cd ~/mozilla/mozilla-beta
hg push

Verify the script Pulled from Aurora into Beta

hg pull -u -r FIREFOX_BETA_54_BASE http://hg.mozilla.org/releases/mozilla-aurora

• Script does version bumps and branding changes

Clobber & push

• Use the Regular Clobbers @ clobberer to clobber all beta builders and then...

# ping Standard8 in #r-d to warn about push
hg push -f

mozilla-central 129 to mozilla-aurora uplift

Don't forget to throttle the Aurora channel

• Use this Bugzilla template to file a bug requesting the disabling of Aurora updates (eg: bug 700794) - until we have sign off from QA
  • Make sure QA is aware (an email to r-d is best)

Script will Tag and commit, continue to push

cd ~/mozilla/mozilla-aurora
hg push

Verify the Diff/Commit of Version Bumps/Branding changes

# check the diffs to ensure nothing additional has been changed
 

• Script will disable profiling changes

Verify the Diff/Commit of mozconfig dtrace & instruments changes (had trouble last time the change manually if needed)

hg diff
hg commit -m "Remove dtrace & instruments in mozconfigs on Aurora as per bug 748669 CLOSED TREE ba=release"
hg out

L10n data changes

# revert to previous Aurora locales - l10n drivers can file post-merge bugs to add new locales
hg revert -r FIREFOX_AURORA_54_END browser/locales/*-locales
hg revert -r FIREFOX_AURORA_54_END mobile/android/locales/*-locales

Verify & Commit l10n changes

$ hg commit -m "Merging in l10n changes NO BUG CLOSED TREE IGNORE BROKEN CHANGESETS ba=release"
$ hg out

Clobber & push

• Use the Regular Clobbers @ clobberer to clobber all aurora builders and then...

# ping Standard8 (or TB driver) in #r-d to warn about push
hg push -f

Completing the Merge

• Email dev.planning and let them know the merge is complete
• Notify #developers
• Notify Release Engineering
• re-open Beta tree (leaving Aurora closed for stabilization)
• Update the merge docs as needed to keep them up to date for next merge day
• Make sure you did file the bug on Aurora update throttling
• Remind Jorge that he needs to prevent an add-on compatibility bump for the hotfix add-ons

Change Template Version Numbers

Now that everything has been bumped edit the following so our Channel Notes and other wiki pages have the correct info and queries now (increment by 1). Also they are very important for daily cronjobs that read the values and nag many many engineers based on triage queries that use them.

• NEXT_VERSION
• CENTRAL_VERSION
• AURORA_VERSION
• BETA_VERSION
• RELEASE_VERSION
• ESR_VERSION (every 7 releases)
• NextReleaseDate
  • Which causes these to auto-update (with the magic of math!)
  • FIREFOX_SHIP_DATE (the next ship date)
  • FIREFOX_MERGE_DATE (the next ship date)
  • CURRENT_CYCLE based upon the calendar

Calculating the current Firefox release # and next release date

The expression reads

floor(Firefox 23 + ((current time - Firefox 23 release date) / 6 weeks))

. This expression counts the number of full 6 week increments since Firefox 23 and adds the total to 23 to give the current release number. Similarly, we can calculate the next release date with

{{#time: Y-m-d | @{{#expr: (3628800 - (({{#time: U | now}} - {{#time: U | 6 Aug 2013}}) mod 3628800)) + {{#time: U | now}}}}}}

. In this case we calculate the time since the Firefox 23 release, take the remainder from dividing by 6 weeks, and subtract that from the current time. This results in the number of days until the next release. Add that to the current time to produce the next release date.