Scratch pad of unusual TODO's for this release:

Profile Guided Optimisation
- we need at least rev1.383 of tinderbox/build-seamonkey-util.pm, so the tinderbox code will need to be retagged (should be doing this anyway)
- cvs update tinderbox to that tag on the three builders
- for each platform that will use PGO (expecting Windows & Linux)
  - bring over the mk_add_options PROFILE_GEN_SCRIPT... line from the nightly mozconfig to the release one, and install the script
  - bring over $ProfiledBuild = 1; from nightly tinder-config.pl to release one
  - compare RAM allocated to nightly and release VM's

Build Engineers

rhelmer

Bonsai queries

Was anything checked in after the "door closed"?

Last checkins:

RC1 2008-??-?? ??:??:??

Bugs

Filed version bump bug in bug 409880

Build data

Type	Build ID	SHA1	Push date	Build machine
[Windows installer]				en-US:fx-win32-1.9-slave2 l10n:fx-win32-1.9-slave2
[Mac compressed]				en-US:fx-mac-1.9-slave2 l10n:fx-mac-1.9-slave2
[Linux compressed]				en-US:fx-linux-1.9-slave2 l10n:fx-linux-1.9-slave2

Notes

RC1

Bootstrap tag: RELEASE_AUTOMATION_M7_1

Setup before starting:

used existing version bump bug to attached diff for bootstrap.cfg and master.cfg
- restarted master
compare nightly and release mozconfig/tinder-config.pl files, merge any desired changes
On fx-mac-1.9-slave2, fx-win32-1.9-slave2, fx-linux-1.9-slave2
- update tinderbox to RELEASE_AUTOMATION_M7_1
  - FIXME update should be automatic - bug 397554
- remove the contents of /builds/verify/*
  - FIXME should be automatically backed up or removed - bug 413178
- restart buildbot
On production-1.9-master
remove the contents of /builds/updates/
remove private and public areas
- rm -rf /data/cltbld/firefox-*
  - FIXME should not be needed - bug 415970
- rm -rf /builds/tags/* - did not do this, plenty of space
  - FIXME should not be needed - bug 415970
- in /home/ftp/pub/firefox/nightly/, remove all except the 3.x-candidates directory for the previous release
  - rm -rf /home/ftp/pub/firefox/nightly/2007*
  - FIXME should not be needed - bug 415970

kick off buildbot (run as cltbld):

buildbot sendchange --username=joduinn --master=localhost:9989 -m "Firefox 3.0 Beta 4 RC1" release

Prestage

had to mkdir /home/cltbld/trunk-automation/prestage/build, not created automatically
- FIXME buildbot bug?

Tag

noticed that linux VMs/slaves are running one hour ahead of Mac, Win32 slaves. This doesnt make sense because we thought they were all ntp-enabled.

Source

had to chmod g-s /home/ftp/ again, looks like it's coming from prestage step
- rhelmer found we're setting it right here! http://mxr.mozilla.org/mozilla/source/tools/buildbot-configs/automation/production-1.9/master.cfg#102
  - FIXED in bug 409880

Build

win32 failed unit tests
- Config::General was not installed
  - installed it
- /builds/logs did not exist
  - created it
- this was an oversight when MozillaBuild was upgraded

mac failed repack
- af was not tagged, so had to checkout a tagged locale so CVS would cache it
  - FIXME should just build all locales bug 415895

linux failed build
- AliveTest could not run, X server was refusing connections
  - ran "xhost +"
    - FIXME slaves should come up usable on boot - bug 417887

Sign

Signing doc

Build declared rc1 broken because of missed updates to mozconfig files as part of bug #407794. Manually bringing release and nightly mozconfig files back in sync, and starting rc2.

L10nVerify

Updates to Test Channel

had to add DisableCompleteJump to rc list and re-push AUS config

verify beta snippets versus betatest snippets
- only URL (download vs. stage) and appv ("3 Beta 3" vs. "3 Beta 3rc3")
- FIXME automate this in bug 409449

cd /opt/aus2/snippets/staging/20080207-Firefox-3.0b3/
find -type d -iregex '.*beta.*' | perl -nle '$a = $_; $a =~ s/beta/betatest/; system("diff -r -u $_ ../20080207-Firefox-3.0b3-test/$a");' 2>&1 | tee /tmp/beta.log

verify beta snippets versus releasetest snippets
- no changes expected
- FIXME automate this in bug 409449

cd /opt/aus2/snippets/staging/20080207-Firefox-3.0b3/
find -type d -iregex '.*beta.*' | perl -nle '$a = $_; $a =~ s/beta/releasetest/; system("diff -r -u $_ ../20080207-Firefox-3.0b3-test/$a");'

verify failed, PatcherConfig made beta like betatest not releasetest. Posted patch in bug 409880, recreated/repushed snippets

ensure that new test snippets are the same as old
- no changes expected

diff -r 20080207-Firefox-3.0b3-test 20080208-Firefox-3.0b3-test

verify beta snippets versus betatest snippets
- only URL (download vs. stage) and appv ("3 Beta 3" vs. "3 Beta 3rc3")
- FIXME automate this in bug 409449

cd /opt/aus2/snippets/staging/20080208-Firefox-3.0b3/
find -type d -iregex '.*beta.*' | perl -nle '$a = $_; $a =~ s/beta/betatest/; system("diff -r -u $_ ../20080208-Firefox-3.0b3-test/$a");' 2>&1 | tee /tmp/beta.log

verify beta snippets versus releasetest snippets
- no changes expected
- FIXME automate this in bug 409449

cd /opt/aus2/snippets/staging/20080208-Firefox-3.0b3/
find -type d -iregex '.*beta.*' | perl -nle '$a = $_; $a =~ s/beta/releasetest/; system("diff -r -u $_ ../20080208-Firefox-3.0b3-test/$a");'

update Verify

results uploaded to https://build.mozilla.org/logs/Firefox_3.0b3_RC3/
- FIXME need to manually remove bump, updateverify config bump only works for rc1 bug 415261

Stage

Sign Installers

Done manually.

pull stage:/data/cltbld/firefox-3.0b2/batch1/stage-signed/

using these installer-signing-instructions here

push signed bits back to same location on stage
complete stage-merged:

# on stage
cd /data/cltbld/firefox-3.0b3/
rsync -av batch1/mar/ stage-merged/
rsync -av batch1/stage-signed/ stage-merged/

Create MD5 and SHA1 checksum files

# on stage
cd /data/cltbld/firefox-3.0b3/stage-merged/
~/bin/checksum-files .

Fix permissions & ownership (on the two SUM files, and the detached sigs)

chown -R cltbld:firefox .
chmod 644 *SUMS

Update Bouncer

Manually add entries to download.mozilla.org

Final Verification

manually verified files under /data/cltbld/firefox-3.0b3/stage-merged/ looked ok.
verified that filename problems from 3.b2 have been fixed. bug 409394

Push to mirrors

push the stage-merged directory to the releases area:

# on stage
rsync -av /data/cltbld/firefox-3.0b3/stage-merged/ /home/ftp/pub/firefox/releases/3.0b3/

Updates to Beta Channel

on Mac build machines, run quick updateverify test for releastest channel

cd /builds/verify/firefox-3.0b3/updates
cat moz19-firefox-*.cfg > releasetest.cfg
sed -i.bak 's/betatest/releasetest/' releasetest.cfg
./verify.sh -t releasetest.cfg 2>&1 | tee releasetest.log
# upload result to build.m.o/logs

Should be all HTTP/200 OK results e.g.:

grep 'HTTP' releasetest.log  | grep -v Found | grep -v 200

Ensure that releasetest channel contents match beta channel contents

cd /opt/aus2/snippets/staging/20080208-Firefox-3.0b3
find -type d -iregex '.*beta.*' | perl -nle '$a = $_; $a =~ s/beta/releasetest/; system("diff -r -u $_ ../20080208-Firefox-3.0b3-test/$a");'

No difs found - which is correct.

Once QA and Website gives formal "go"

# put snippets on beta
$ sudo su - cltbld
$ cd /opt/aus2/snippets/staging
$ ~/bin/pushsnip 20080208-Firefox-3.0b3

on Mac build machines, run quick updateverify test for beta channel

cd /builds/verify/firefox-3.0b3/updates
cat moz19-firefox-*.cfg > beta.cfg
sed -i.bak 's/betatest/beta/' beta.cfg
./verify.sh -t beta.cfg 2>&1 | tee beta.log
# upload result to build.m.o/logs