Thunderbird 2.0.0.14:BuildNotes

Build Engineers

nthomas, rick

Version/config bump bug

bug 426259 Morphed from the cancelled Tb2.0.0.13 build

Bonsai queries

Checking for no changes on the rel branch since Fx20014

• cvsroot
• l10n

Tags

Using the GECKO181_20080311_RELBRANCH.

Module	Tag	Pull date
cvsroot/mozilla	THUNDERBIRD_2_0_0_14_RELEASE	GECKO181_20080311_RELBRANCH @ 2008-04-17 13:27 PDT
	THUNDERBIRD_2_0_0_14_RC1	GECKO181_20080311_RELBRANCH @ 2008-04-17 13:27 PDT
l10n/l10n	THUNDERBIRD_2_0_0_14_RELEASE	GECKO181_20080311_RELBRANCH @ 2008-04-17 13:42 PDT
	THUNDERBIRD_2_0_0_14_RC1	GECKO181_20080311_RELBRANCH @ 2008-04-17 13:42 PDT

NB: Talkback not listed, we're using pre-built binaries (bug 403701). It was still tagged because this change is very new, and the automation needs remedial lessons.

Build data

See page history for BuildIDs of prior RCs

Type	Build ID	SHA1	Push date	Build machine
[Windows installer]	RC1:			patrocles (cerberus-vm for l10n)
[Mac compressed]	RC1:			xserve02
[Linux compressed]	RC1:			crazyhorse (karma for l10n)

Automation

Bootstrap tag: RELEASE_AUTOMATION_M8_1

Setup before starting:

• used existing version bump bug to attached diff for tb-moz18-bootstrap.cfg
• revert mozilla/config/milestone.txt and mozilla/client.mk on the rel branch so that Tagging can substitute cleanly (FIXME: need bug)
• ensure that machines have enough resources (bug 393274)
  • On production-1.8-master, remove private and public areas:
    • rm -rf /data/cltbld/firefox-* (check with folks responsible for the releases being rm'd before removing ... in this case it was nthomas and bhearsum)
    • rm -rf /builds/tags/* - did not do this, plenty of space (and handy to have copies of what was tagged to work around CVS's lack of atomicity of tagging)
    • in /home/ftp/pub/firefox/nightly/, remove all except the 2.0.0.x-candidates directory for the previous release
    • pull in 2.0.0.12 (previous release) files with rsync -av stage.mozilla.org:/home/ftp/pub/thunderbird/nightly/2.0.0.12-candidates /data/ftp/pub/thunderbird/nightly/
  • On the build machines (see table above), checked that there was sufficient working space (~ 1GB), and stopped tinderbox. Also verified there is space available on bm-xserve04 for l10n-verify.
  • On prometheus-vm, checked that there was space for another update run (4GB)

The following bits, to the end of this Automation section, still need to be done. Delayed due to ssh key issues on the various tinderbox clients. Will do the Tag and Source steps tho. 2008-04-17.

• on each machine, setup release automation using the tag above

cd /builds/release   # or /cygdrive/X/builds/release
cvs -nq up -r RELEASE_AUTOMATION_M8_1
# check for merges (M) or conflicts (C); if any found, check for local changes
cvs -q up -r RELEASE_AUTOMATION_M8_1
cd configs
cvs up -A     # in case we need to make changes later
cd ..
rm bootstrap.cfg logs
ln -s configs/tb-moz18-bootstrap.cfg bootstrap.cfg
mkdir -p old-logs/logs.tb20014/rc1
ln -s old-logs/logs.tb20014/rc1 logs

make test

Some machines don't have the perl module Config::General so comment out

use Config::General;

at the top of Bootstrap/Step/PatcherConfig.pm. Only need this on prometheus-vm anyway, which has it.

Tag

On production-1.8-master in /builds/release:

./release -o Tag 2>&1 | tee logs/release-Tag-RC1.log

Source

On production-1.8-master in /builds/release:

./release -o Source 2>&1 | tee logs/release-Source-RC1.log

Failed with message

build-console.build.mozilla.org - Step Source died: No such config variable:
externalStagingUser at Bootstrap/Config.pm line 93.

Appears to be related to this configuration change which removed the externalStagingUser variable.

However, tools/release/Bootstrap/Util.pm on the RELEASE_AUTOMATION_M8_1 tag still references that variable. There were changes made on the trunk for Util.pm that removed those references.

Resolved with this commit which restored the external* variables.

This change will need to be undone once the thunderbird 2.0.0.14 release is complete.

The Source step was completed by doing the following on production-1.8-master

 cd /builds/release/configs
 cvs up
 cd ..
  ./release -o Source -p 2>&1 | tee logs/release-Source-RC1-Push.log

This completed successfully.

Build

There ssh key issues to workaround - the nightly builders were converted to tbirdbld but releases still use cltbld. On each box,

• shut down tinderbox builds and add a comment on the Mozilla1.8 tree
• in ~/.ssh

rm id_dsa; ln -s cltbld_dsa id_dsa
rm id_dsa.pub; ln -s cltbld_dsa.pub id_dsa.pub

• update /builds/tinderbox/Tb-Mozilla1.8-Release/tinderbox-configs/CVS/Root so that it uses the cltbld account (just for this release, we'll leave it on cltbld)
• update Bootstrap as in the Automation section

On Windows, check that lines 43-49 of Build.pm are commented out, since Cygwin complains "Permissied denied" trying to execute those lines. The disks are already mounted in binmode.

Then start the build, by changing directory to /builds/release (prefix with /cygdrive/e/ on Windows) and

./release -o TinderConfig 2>&1 | tee logs/release-TinderConfig.log
./release -o Build 2>&1 | tee logs/release-Build.log

After the build is done, and if the tinderbox is not needed for l10n Repacks (eg patrocles, crazyhorse),

• reverse the changes in ~/.ssh. NB: bm-xserve02 uses ffxbld, others use tbirdbld. Eg

rm id_dsa; ln -s tbirdbld_dsa id_dsa
rm id_dsa.pub; ln -s tbirdbld_dsa.pub id_dsa.pub

• restart tinderbox

cd /builds/tinderbox
./tinderbox multi start

Repack

On the tinderboxes in the table above

• stop tinderbox
• do the ssh key changes as in the Build section
• update Bootstrap as in the Automation section

./release -o Repack -e 2>&1 | tee logs/release-Repack-Execute.log
./release -o Repack -p 2>&1 | tee logs/release-Repack-Push.log
./release -o Repack -a 2>&1 | tee logs/release-Repack-Announce.log

	karma	bm-xserve02	cerberus-vm
Repack -e	ok	ok	ok
Repack -p	ok	not ok file has vanished: "/home/ftp/pub/thunderbird/nightly/2.0.0.14-candidates/rc1/.thunderbird-2.0.0.14.pa-IN.linux-i686.complete.mar.KwvZoP"	not ok rsync: link_stat "/data/cltbld/thunderbird-2.0.0.12/." failed: No such file or directory (2) rsync error: some files could not be transferred (code 23) at main.c(702)
Repack -a	ok

After win32 l10n, had to set cerberus-vm back to text mode

mount -t -sc /cygdrive

so that tinderbox would run. This box was very slow, taking 7 hours to complete the repack.

Undo the ssh changes like Build when they are done.