Releases/Thunderbird 2.0.0.21/BuildNotes

From MozillaWiki
Jump to: navigation, search

Build Engineers

nthomas
Version/config bump/tracking in bug 480864

Bonsai queries

Build 1: cvsroot

Tags

Build 1:

Module Parent Branch Created Tag/Branch <thead> </thead> <tbody> </tbody>
style="background:#efefef" Pull date
rowspan="2" cvsroot/mozilla
MOZILLA_1_8_BRANCH
GECKO181_20090302_RELBRANCH
2009-03-01 16:14 PST
-
GECKO181_20090302_RELBRANCH
THUNDERBIRD_2_0_0_21_RELEASE, THUNDERBIRD_2_0_0_21_BUILD1
2009-03-02 14:49 PST (but see Tag below)
-
rowspan="2" l10n/l10n
MOZILLA_1_8_BRANCH
GECKO181_20090302_RELBRANCH
2009-03-01 16:14 PST
-
GECKO181_20090302_RELBRANCH
THUNDERBIRD_2_0_0_21_RELEASE, THUNDERBIRD_2_0_0_21_BUILD1
2009-03-02 15:18 PST
}

Build data

Type <thead> </thead> <tbody> </tbody>
style="background:#efefef" Build machine
[Windows installer/zip]
production-patrocles
-
[Mac compressed]
bm-xserve05
-
[Linux compressed]
production-crazyhorse
}

Notes

Build 1

Using RELEASE_AUTOMATION_M13, same as Fx3.0.7 but new for Tb. Leaving Tinderbox on RELEASE_AUTOMATION_M11 (don't need bug bug 413491).

Slave cleanup

  • production-prometheus-vm
    • need ~10GB for tag, source, update generation and linux verify, stage; removed /builds/updates/thunderbird-2.0.0.19 to leave 14GB free
    • 'ssh -i .ssh/ffxbld_dsa ffxbld@hg.mozilla.org' to accept key fingerprint (one off)
  • production-crazyhorse
      • 21GB free on /builds
      • update tinderbox to automation tag
        • cd /builds/tinderbox/mozilla/tools/tinderbox; cvs up -CPd -r RELEASE_AUTOMATION_M11
  • bm-xserve05
      • need ~20GB for mac builds, l10n_verify, mac update verify (but 13GB already allocated for builds); removed /builds/verify/thunderbird-2.0.0.19/ to leave 15G free
      • update tinderbox to automation tag
        • cd /builds/tinderbox/mozilla/tools/tinderbox; cvs up -CPd -r RELEASE_AUTOMATION_M11
      • 'ssh -i .ssh/ffxbld_dsa ffxbld@hg.mozilla.org' to accept key fingerprint (one off)
  • production-patrocles
      • 50GB free
      • update tinderbox to automation tag
        • cd /cygdrive/e/builds/tinderbox/mozilla/tools/tinderbox; cvs up -CPd -r RELEASE_AUTOMATION_M11
  • production-pacifica-vm
      • 30GB free, plenty for update verify
      • 'ssh -i .ssh/ffxbld_dsa ffxbld@hg.mozilla.org' to accept key fingerprint (one off)
  • Land changes and retag tb-moz18-bootstrap.cfg
      • cvs tag -d RELEASE_AUTOMATION_M13 tb-moz18-bootstrap.cfg # to delete the tag on the old version
      • cvs tag RELEASE_AUTOMATION_M13 tb-moz18-bootstrap.cfg # to tag the new version

Setup master before starting:

  • update mozilla/tools/buildbot-configs/automation/production/tb-master.cfg for most recent milestone of bootstrap
  • as buildmaster@production-1.8-master, cd /builds/buildbot/Thunderbird-Automation/buildbot-configs. Check for local diff, revert any changes from last release, dry run a cvs up, then cvs up
  • master.cfg is "Locally Modified" to contain passwords; identical to repository version otherwise.
  • Ran 'buildbot reconfig /home/buildmaster/Thunderbird-Automation'
  • Buildbot waterfall is here (MPT-VPN); this is a separate instance from the Firefox master, with production-prometheus-vm and bm-xserve05 running a slave for each master
  • Stop nightly builds on bm-xserve05 (~cltbld $ buildbot stop /builds/slave/) and annotate Mozilla1.8 tree to explain its absence

Start automation run:

buildbot sendchange --username=nthomas --master=localhost:9990 -m"Thunderbird 2.0.0.21 build 1" release

Tag

config/milestone.txt was not bumped (we expect Firefox to have already done this). Manual fix (done between /cvsroot and /l10n tagging)

cvs up -r GECKO181_20090302_RELBRANCH milestone.txt
# edit and remove pre to have '1.8.1.21'
cvs ci -m "Bug 480864, update milestone.txt for Tb2.0.0.21 (on GECKO181_20090302_RELBRANCH)" milestone.txt
cvs tag -F THUNDERBIRD_2_0_0_21_BUILD1 milestone.txt
cvs tag -F THUNDERBIRD_2_0_0_21_RELEASE milestone.txt

bug 481079 to fix this up.

Source

Automated, no problems

Build/Repack

Automated, no problems

Sign

l10n verification

Failed with

Step Repack died: No such config variable: hgToolsTag at Bootstrap/Config.pm line 93.

Added 'hgToolsTag = default' to tb-moz18-bootstrap.cfg, and moved the RELEASE_AUTOMATION_M13 up to the new revision. No problems on the second attempt.

Generate updates

Automated, no problems

Publish updates to test channels (betatest & releasetest)

Automated, no problems

Update verify

Mac & Windows bombed out when pulling build/tools from hg - need to investigate. Stopped Linux because all the update verify configs are missing 2.0.0.16/17/18.

Coop fixed the configs, restarted all three platforms. Win32 bombed out with key problem pushing config change back to hg.m.o, fixed hgSshKey in tb-moz18-bootstrap.cfg and 3 other 1.8 configs and restarted win32.

All were green.

Push updates to beta channel

$ sudo su - cltbld

$ cd /opt/aus2/snippets/staging
$ time ~/bin/backupsnip 20090302-Thunderbird-2.0.0.21-beta
real    41m7.376s
user    0m37.186s
sys     1m5.541s

Waited for formal "go" to beta.

$ sudo su - cltbld
$ cd /opt/aus2/snippets/staging
$ time ~/bin/pushsnip 20090302-Thunderbird-2.0.0.21-beta
real    1m24.075s
user    0m0.099s
sys     0m5.007s

Sign installers

  • Done manually using these installer-signing-instructions here
  • complete stage-merged:
# on stage
cd /data/cltbld/thunderbird-2.0.0.21/
rsync -av batch1/mar/ stage-merged/
rsync -av batch1/stage-signed/ stage-merged/
  • Create MD5 and SHA1 checksum files
# on stage
cd /data/cltbld/thunderbird-2.0.0.21/stage-merged/
~/bin/checksum-files .
  • Fix permissions & ownership (on the two SUM files, and the detached sigs)
chown -R cltbld:thunderbird .
chmod 644 *SUMS

Update bouncer

  • Manually added entries to download.mozilla.org
  • Note for next release: Do not remove the Check Now bit on the Thunderbird-2.0.0.21 Products until well after the change to the rsync module (to prevent the likes of bug 464566)

Push to mirrors

  • push the stage-merged directory to the releases area:
# on stage (as cltbld)
rsync -av /data/cltbld/thunderbird-2.0.0.21/stage-merged/ /home/ftp/pub/thunderbird/releases/2.0.0.21/

Update rsync module

On stage.mozilla.org, add the new release to the smaller, faster, rsync module:

# on stage (as cltbld)
cd /pub/mozilla.org/zz/
vi rsyncd-mozilla-current.exclude 

and replace the previous latest release (Tb2.0.0.19) with:

+ thunderbird/releases/2.0.0.21
+ thunderbird/releases/2.0.0.21/**

Final verification

  • Verify that releasetest points to valid bouncer links:
hg clone http://hg.mozilla.org/build/tools
cd tools/release/updates
cat moz18-thunderbird-*.cfg | grep -v major | sed 's/betatest/releasetest/' | grep -v 2.0a | grep -v 2.0b > update.cfg
./verify.sh -t update.cfg 2>&1 | tee quickVerify.log
  • Look for any HTTP error codes besides 200 ("OK") and 302 ("Found"):
grep HTTP quickVerify.log | grep -v 200 | grep -v 302

Found that www.mirrorservice.org was returning 404s. Turns out there are 6 machines in DNS for this and one of them was stale. Can't raise the admin on irc or email, disabled in bouncer.

  • Before pushing final updates,verify that "release" and "releasetest" channel match:
    1. on aus2-staging
    cd /opt/aus2/snippets/staging/20090302-Thunderbird-2.0.0.21 find -type d -iregex '.release.' | perl -nle '$a = $_; $a =~ s/release/releasetest/; system("diff -r -u $_ ../20090302-Thunderbird-2.0.0.21-test/$a");'

Publish updates to release channel

  • In case you need to cancel/abort updates, the quickest way to do this is here
  • While waiting for QA to finish testing, do a backup:
# on aus2-staging verify we have correct tools
$ sudo su - cltbld
$ cd ~/bin
$ cvs update -dP
# backup snippets on release/live channel
$ time ~/bin/backupsnip 20090302-Thunderbird-2.0.0.21
real    43m45.645s
user    0m36.126s
sys     0m47.343s
  • After QA finished testing the releasetest channel and go from release driver, enable release channel:
    1. on aus2-staging
    2. put snippets on release/live channel
    $ sudo su - cltbld $ time ~/bin/pushsnip 20090302-Thunderbird-2.0.0.21 real 1m19.576s user 0m0.082s sys 0m3.777s

We disabled updates because of problematic redirects of first run pages, part of moving content to www.mozillamessaging.com, and then enabled them after that was fixed.

Update symlink

On stage.m.o, change symlink to point to new release:

$ cd /pub/mozilla.org/thunderbird/releases
# confirm where latest symlinks point to where before starting
$ ls -la latest*
lrwxrwxrwx  1 cltbld thunderbird 8 Dec 30 11:44 latest -> 2.0.0.19
lrwxrwxrwx  1 cltbld thunderbird 8 Dec 30 11:44 latest-2.0 -> 2.0.0.19
$ rm latest    # (note no trailing slash)
$ ln -s 2.0.0.21 latest
$ rm latest-2.0   # (note no trailing slash)
$ ln -s 2.0.0.21 latest-2.0
# confirm latest symlinks point to newest release
$ ls -la latest*
lrwxrwxrwx  1 cltbld thunderbird 8 Mar 18 13:54 latest -> 2.0.0.21
lrwxrwxrwx  1 cltbld thunderbird 8 Mar 18 13:54 latest-2.0 -> 2.0.0.21