Releases/Firefox 3.0.2:BuildNotes

Build Engineers

joduinn
Version/config bump bug

Bonsai queries

Updated CVS Tags devmo page.

Tags

Build 1:

Module	Branch	Tag	Pull date
cvsroot/mozilla	HEAD	FIREFOX_3_0_2_BUILD1	2008-08-27 01:31 PDT
l10n/l10n	HEAD	FIREFOX_3_0_2_BUILD1	2008-08-27 01:31 PDT

Build 2:

Module	Branch	Tag	Pull date
cvsroot/mozilla	GECKO190_20080827_RELBRANCH	FIREFOX_3_0_2_BUILD2	N/A
l10n/l10n	GECKO190_20080827_RELBRANCH	FIREFOX_3_0_2_BUILD2	N/A

Build 3:

Module	Branch	Tag	Pull date
cvsroot/mozilla	GECKO190_20080827_RELBRANCH	FIREFOX_3_0_2_BUILD3	N/A
l10n/l10n	GECKO190_20080827_RELBRANCH	FIREFOX_3_0_2_BUILD3	N/A

Build 4:

Module	Branch	Tag	Pull date
cvsroot/mozilla	GECKO190_20080827_RELBRANCH	FIREFOX_3_0_2_BUILD4	N/A
l10n/l10n	GECKO190_20080827_RELBRANCH	FIREFOX_3_0_2_BUILD4	N/A

Build 5:

Module	Branch	Tag	Pull date
cvsroot/mozilla	GECKO190_20080827_RELBRANCH	FIREFOX_3_0_2_BUILD5	N/A
l10n/l10n	GECKO190_20080827_RELBRANCH	FIREFOX_3_0_2_BUILD5	N/A

Build 6:

Module	Branch	Tag	Pull date
cvsroot/mozilla	GECKO190_20080827_RELBRANCH	FIREFOX_3_0_2_BUILD6	N/A
l10n/l10n	GECKO190_20080827_RELBRANCH	FIREFOX_3_0_2_BUILD6	N/A

Build data

Type	Build ID	SHA1	Push date	Build machine
[Windows installer/zip]				fx-win32-1.9-slave2
[Mac compressed]				fx-mac-1.9-slave2
[Linux compressed]				fx-linux-1.9-slave2

Notes

Build 1

Bootstrap Tag: RELEASE_AUTOMATION_M11

Setup before starting:

• Updated master.cfg, bootstrap.cfg. Retagged bootstrap.cfg.
• On the slaves:
  • ran 'DISPLAY=:0 xhost +' to make sure linux AliveTest works.
  • remove /builds/verify/firefox-3.0.1
  • update tinderbox to RELEASE_AUTOMATION_M10
    • For win32:

cd /e/builds/tinderbox/mozilla/tools/tinderbox
cvs up -r RELEASE_AUTOMATION_M11
for dir in fx19rel fx19l10nrel xr19rel; do 
  cd /e/$dir; 
  for f in `ls *.pl *.html INSTALL README | grep -v tinder-config.pl`; do 
    cp -pv /e/builds/tinderbox/mozilla/tools/tinderbox/$f .;
  done
  cp -pv post-mozilla-rel.pl post-mozilla.pl
done

• • On the linux slave, removed:
    • /builds/updates/firefox-3.0.1 and /builds/updates/3.0.2-test
    • /builds/source/*
    • /data/cltbld/thunderbird-3.0a2*
  • On the mac slave, removed:
    • /builds/tinderbox/Tb-Mozilla1.9-Release/Darwin_8.8.4_Depend
    • /builds/tinderbox/Tb-Mozilla1.9-l10n-Release/Darwin_8.8.4_Depend
    • /builds/tinderbox/Xr-Mozilla1.9-Release/Darwin_8.8.4_Depend

• Space on slaves before starting:
  • fx-linux-1.9-slave2: 24G on /builds
  • fx-mac-1.9-slave2: 24G on /
  • fx-win32-1.9-slave2: 17G on d:, 5G on e: (disk heavy work is done on d, eg update_verify. build/repack is done on e, but mostly just overwrites existing data)

• Gulped, and kicked off automation:

buildbot sendchange --username=bhearsum --master=localhost:9989 -m"Firefox 3.0.2build1 release" gogogo

Tag

Automated, no problems.

Source

Automated, no problems.

Build 2

Code errors were found after we tagged for build 1, therefore we had to spin a build 2.

Bootstrap Tag: RELEASE_AUTOMATION_M11

• Landed bootstrap bumps for the respin

Tag

Automated, no problems.

Source

Automated, no problems

Build

Automated, no problems.

Repack

No problems on linux and mac. Win32 failed out in Repack::Execute because the E: drive was full. It had finished the repacks, but failed to push to stage. Cleared space (deleted the Tb and Xr release build directories) and restarted from Repack::Execute.

Sign

• Signing bailed out during 'ro' with this error:

*** Resign list hit; resigning /home/cltsign/signing-work/firefox-3.0.2/tmp/RepackExe-JeG9pR/localized/uninstall/helper.exe...
Signing /home/cltsign/signing-work/firefox-3.0.2/tmp/RepackExe-JeG9pR/localized/uninstall/helper.exe...
Error: Invalid timestamp http address
Error: TimeStamping Failed.  Result = 80070001, (-2147024895)
Failed to sign /home/cltsign/signing-work/firefox-3.0.2/tmp/RepackExe-JeG9pR/localized/uninstall/helper.exe; bailing

Build 3

Tag

No Problems

Source

No problems

Build

No problems

Repack

No problems

Sign

no problems

L10nVerify

L10nverify spewed tons of output from updater.ini differences. It turns out that https://bugzilla.mozilla.org/show_bug.cgi?id=446527 busted updater.ini on linux and mac. There was win32 about as well, which was completely legitimate (because Locale= was newly added to updater.ini for win32).

Respun for the linux & mac problem.

Build 4

Tag

• Automated, no problems.

Source

• Automated, no problems.

Build

• Automated, no problems.

Repack

• Automated, no problems.

Sign

instructions
Hit timestamp problem again. To work-around, did the following (from https://bugzilla.mozilla.org/show_bug.cgi?id=415966#c25):

mkdir signed-build4-partial unsigned-build4-partial
cp signed-build4/*en-US* signed-build4-partial/
cp unsigned-build4/*en-US* unsigned-build4-partial/
# copied everything from it -> zh-TW from unsigned-build4 to unsigned-build4-partial
perl ./signing/sign-release.pl -o signed-build${BUILD}-partial -M dir -d unsigned-build${BUILD}-partial -a ${PRODUCT} --release ${TAG} 2>&1 | tee -a win32_signing_build${BUILD}.log
# to merge the signed builds back together
rsync -av signed-build4-partial/ signed-build4/
# then push the builds back to stage (instructions on the signing doc)

L10nVerify

Some string changes in updater.ini - this is legitimate.

Generate Updates

• updates::execute has been run, automation needs to be restarted from updates::push (after 2.0.0.17 test snippets are done being pushed)
• locally modified master.cfg to skip over updates::execute and start at updates::push. Now in progress.

Publish Updates to Test Channels (betatest & releasetest)

• manually forced step from waterfall. No problems, worked fine.

Stage

• manually forced step from waterfall. No problems, worked fine.

respin declared because of bug 453227

Build5

Manual cleanup skipped. Had fun restarting master because master.cfg was accidently refreshed from cvs earlier this morning, losing all the passwords. Logged into production-1.9-master, as buildbot, and ran the following:

buildbot sendchange --username=joduinn --master=localhost:9989 -m"Firefox 3.0.2build5 release" gogogo

Tag

• Automated, no problems.

Source

• Automated, no problems.

Build

• Automated, no problems.

Repack

• Automated, no problems.

Sign

instructions

L10nVerify

• Automated, no problems. Expected fails for gu-IN during the FF3.0betas

Generate Updates

• Automated, no problems.

Publish Updates to Test Channels (betatest & releasetest)

• Automated, no problems.

Update Verify

• Automated, no problems.

Stage

• Automated, no problems.

Push to beta channel

While waiting for QA to give "ok", on aus2-staging do:

 # put snippets on beta
 $ sudo su - cltbld
 # make sure using latest version of scripts in mozilla/tools/release/bin/
 $ cd bin
 $ cvs update . 
 $ cd /opt/aus2/snippets/staging
 # note the required parameter must match what will be used with pushsnip below. 
$  time ~/bin/backupsnip 20080908-Firefox-3.0.2-beta
Running /bin/tar cfvj /opt/aus2/snippets/backup/20080910-1-pre-20080908-Firefox-3.0.2-beta.tar.bz2 .
real    30m49.454s
user    0m34.069s
sys     0m37.876s
$
$ time ~/bin/pushsnip 20080908-Firefox-3.0.2-beta
sent 858677 bytes  received 49674 bytes  21372.96 bytes/sec
total size is 687093  speedup is 0.76
real    0m42.322s
user    0m0.051s
sys     0m4.122s

Respin declared for bug 454406, bug 443918 and bug 455283

Build6

Manual cleanup skipped. Logged into production-1.9-master, as buildbot, and ran the following:

buildbot sendchange --username=joduinn --master=localhost:9989 -m"Firefox 3.0.2build6 release" pleasenomorerespins

Tag

• Automated, no problems.

Source

• Automated, no problems.

Build

• Automated, no problems.

Repack

• Automated, no problems.

Sign

instructions

Signing failed out on 'de' with:

 Error: Invalid timestamp http address
 Error: TimeStampign Failed. Result = 80070001, (-2147024895)

To work-around, did the following (from https://bugzilla.mozilla.org/show_bug.cgi?id=415966#c25):

mkdir signed-build6-partial unsigned-build6-partial
cp signed-build6/*en-US* signed-build6-partial/
cp unsigned-build6/*en-US* unsigned-build6-partial/
# copied everything from 'de' -> 'zh-TW' from unsigned-build6 to unsigned-build6-partial
perl ./signing/sign-release.pl -o signed-build${BUILD}-partial -M dir -d unsigned-build${BUILD}-partial -a ${PRODUCT} --release ${TAG} 2>&1 | tee -a win32_signing_build${BUILD}.log
# to merge the signed builds back together
rsync -av signed-build6-partial/ signed-build6/
# then push the builds back to stage (instructions on the signing doc)

Generate Updates

Need to bring people who already updated to 3.0.2 build5 up to build6, so do this (after the automated run)

# connect as cltbld@fx-linux-1.9-slave2
cd /builds/updates/firefox-3.0.2/patcher/temp/firefox/3.0.1-3.0.2
for dir in aus2.test aus2.beta aus2; do
  # copy complete snippets from 3.0.1 into a new dir for 3.0.2
  mkdir -p b5-b6-${dir}/Firefox
  rsync -a --exclude 'partial.txt' ${dir}/Firefox/3.0.1/ b5-b6-${dir}/Firefox/3.0.2/
  # use the buildid's from build5
  mv b5-b6-${dir}/Firefox/3.0.2/Darwin_Universal-gcc3/2008070206  b5-b6-${dir}/Firefox/3.0.2/Darwin_Universal-gcc3/2008090512
  mv b5-b6-${dir}/Firefox/3.0.2/Linux_x86-gcc3/2008070206         b5-b6-${dir}/Firefox/3.0.2/Linux_x86-gcc3/2008090512
  mv b5-b6-${dir}/Firefox/3.0.2/WINNT_x86-msvc/2008070208         b5-b6-${dir}/Firefox/3.0.2/WINNT_x86-msvc/2008090514
done
rsync -a -e 'ssh -i ~/.ssh/aus' b5-b6-aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20080919-Firefox-3.0.2build5-build6-test
rsync -a -e 'ssh -i ~/.ssh/aus' b5-b6-aus2.beta/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20080919-Firefox-3.0.2build5-build6-beta
rsync -a -e 'ssh -i ~/.ssh/aus' b5-b6-aus2/      cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20080919-Firefox-3.0.2build5-build6

Then go to aus2-staging and push 20080919-Firefox-3.0.2build5-build6-test live.

Push to beta channel

NB: Be sure to push both 20080917-Firefox-3.0.2-beta and 20080919-Firefox-3.0.2build5-build6-beta directories.

Publish Updates to Release Channel

NB: Be sure to push both 20080917-Firefox-3.0.2 and 20080919-Firefox-3.0.2build5-build6 directories.