Firefox 3.0b3:BuildNotes

Revision as of 01:22, 8 February 2008 by Rhelmer (talk | contribs) (correction on expected results)

Build Engineers

rhelmer

Bonsai queries

Was anything checked in after the "door closed"?

Last checkins:

Bugs

Filed version bump bug in bug 409880

Tags

Module Tag Pull date
cvsroot/mozilla
l10n/l10n

Build data

Type Build ID SHA1 Push date Build machine
[Windows installer/zip] en-US:fx-win32-1.9-slave2 l10n:fx-win32-1.9-slave2
[Mac compressed] en-US:fx-mac-1.9-slave2 l10n:fx-mac-1.9-slave2
[Linux compressed] en-US:fx-linux-1.9-slave2 l10n:fx-linux-1.9-slave2

Notes

RC1

Bootstrap tag: RELEASE_AUTOMATION_M7_1

Setup before starting:

  • used existing version bump bug to attached diff for bootstrap.cfg and master.cfg
    • restarted master
  • compare nightly and release mozconfig/tinder-config.pl files, merge any desired changes
  • ensure that machines have enough resources
    • FIXME should be automatically backed up or removed - bug needed
  • On fx-mac-1.9-slave2, fx-win32-1.9-slave2, fx-linux-1.9-slave2
    • update tinderbox to RELEASE_AUTOMATION_M7_1
    • remove the contents of /builds/verify/*
      • FIXME should be automatically backed up or removed - bug needed
    • restart buildbot
  • On production-1.9-master, remove private and public areas:
  • remove the contents of /builds/updates/
    • FIXME need to do this on each RC or updates will fail - bug needed
    • rm -rf /data/cltbld/firefox-*
      • FIXME should be automatically backed up or removed - bug needed
    • rm -rf /builds/tags/* - did not do this, plenty of space
      • FIXME should be automatically backed up or removed - bug needed
    • in /home/ftp/pub/firefox/nightly/, remove all except the 3.x-candidates directory for the previous release
      • rm -rf /home/ftp/pub/firefox/nightly/2007*
        • FIXME should be automatically backed up or removed - bug needed
  • kick off buildbot (run as cltbld):
buildbot sendchange --username=rhelmer --master=localhost:9989 -m "Firefox 3.0 Beta 3 RC1" release

Prestage

  • had to mkdir /home/cltbld/trunk-automation/prestage/build, not created automatically

Tag

  • noticed that linux VMs/slaves are running one hour ahead of Mac, Win32 slaves. This doesnt make sense because we thought they were all ntp-enabled.

Source

Build

  • win32 failed unit tests
    • Config::General was not installed
      • installed it
    • /builds/logs did not exist
      • created it
    • not sure how we hit these, was slave rebuilt since 3.0b2?!
  • mac failed repack
    • af was not tagged, so had to checkout a tagged locale so CVS would cache it
      • FIXME should just build all locales, link to existing bug
  • linux failed build
    • AliveTest could not run, X server was refusing connections
      • ran "xhost +"

Sign

Signing doc

  • Build declared rc1 broken because of missed updates to mozconfig files as part of bug #407794. Manually bringing release and nightly mozconfig files back in sync, and starting rc2.

RC2

Bootstrap tag: RELEASE_AUTOMATION_M7_1

Setup before starting:

  • used existing version bump bug to attached diff for bootstrap.cfg and master.cfg
    • restarted master
  • compare nightly and release mozconfig/tinder-config.pl files, merge any desired changes
  • ensure that machines have enough resources
    • FIXME should be automatically backed up or removed - bug needed
  • On fx-mac-1.9-slave2, fx-win32-1.9-slave2, fx-linux-1.9-slave2
    • update tinderbox to RELEASE_AUTOMATION_M7_1
    • remove the contents of /builds/verify/*
      • FIXME should be automatically backed up or removed - bug needed
    • restart buildbot
  • noticed that linux VMs/slaves are running one hour ahead of Mac, Win32 slaves. This doesnt make sense because we thought they were all ntp-enabled.
    • fixed manually and synced with hwclock
  • kick off buildbot (run as cltbld):
buildbot sendchange --username=rhelmer --master=localhost:9989 -m "Firefox 3.0 Beta 3 RC2" release

Prestage

Tag

Source

Build

  • set year incorrectly doing manual fix for Linux VMs
    • fixed and restarted Linux slave and master
    • master restart forced Win32 build restart
  • Linux clobber not working; kernel update not reflected in bootstrap.cfg
    • clobbered manually and restarted
  • Mac and Win32 failed to upload breakpad symbols bug 415913
    • fixed by rerunning "make uploadsymbols" with key set correctly
    • tinderbox should visibly break when this happens
    • FIXME this is due to the path to the key not set correctly in bootstrap.cfg, bug needed

Sign

Signing doc

RC3

Bootstrap tag: RELEASE_AUTOMATION_M7_1

Calling this RC3, but really it's RC2 with extra locales and win32 re-signed.

Setup before starting:

  • used existing version bump bug to attached diff for bootstrap.cfg and master.cfg
    • also l10n_release mozconfigs
    • comment out build and 3.0b3 clean steps
    • restarted master
  • landed new shipped-locales on relbranch
  • forced mozconfigs to only build locales we want
  • on master, copy rc2 en-US to rc3
mv rc2 rc3
cd rc3
rm -rf *win32* *partial.mar *source* *.log
  • On fx-mac-1.9-slave2, fx-win32-1.9-slave2, fx-linux-1.9-slave2
    • remove the contents of /builds/verify/*
      • FIXME should be automatically backed up or removed - bug needed
    • restart buildbot
  • kick off buildbot (run as cltbld):
buildbot sendchange --username=rhelmer --master=localhost:9989 -m "Firefox 3.0 Beta 3 RC2" release

Prestage

Tag

  • manually pulled in locales:
    • es-ES ja ja-JP-mac pa-IN
    • used timestamp 2008-02-06 03:24 and merged to GECKO19b3_20080205_RELBRANCH
cvs co l10n
cvs up -D 2008-02-06 03:30 PST es-ES ja ja-JP-mac
cvs tag -b GECKO19b3_20080205_RELBRANCH es-ES ja ja-JP-mac

Source

Build

Sign

Signing doc

L10nVerify

Updates to Test Channel

  • had to add DisableCompleteJump to rc list and re-push AUS config
  • verify beta snippets versus betatest snippets
    • only URL (download vs. stage) and appv ("3 Beta 3r" vs. "3 Beta 3rc3")
    • FIXME automate this in bug 409449
cd /opt/aus2/snippets/staging/20080206-Firefox-3.0b3/
find -type d -iregex '.*beta.*' | perl -nle '$a = $_; $a =~ s/beta/betatest/; system("diff -r -u $_ ../20080206-Firefox-3.0b3-test/$a");' 2>&1 | tee /tmp/beta.log
  • verify beta snippets versus releasetest snippets
    • no changes expected
    • FIXME automate this in bug 409449
cd /opt/aus2/snippets/staging/20080206-Firefox-3.0b3/
find -type d -iregex '.*beta.*' | perl -nle '$a = $_; $a =~ s/beta/releasetest/; system("diff -r -u $_ ../20080206-Firefox-3.0b3-test/$a");'

update Verify

Stage

Sign Installers

Update Bouncer

Final Verification

Push to mirrors

Updates to Beta Channel

Wall clock timings