Releases/Firefox 3.6.14/BuildNotes

Build Engineers

armenzg - bug 624496

Signed-off Revision(s)

28ce03bbd37c

Tags

Build #	Tag	Changeset
1	FIREFOX_3_6_14_BUILD1 FIREFOX_3_6_14_RELEASE	http://hg.mozilla.org/releases/mozilla-1.9.2/rev/69bfc730b3e9

Bugs hit

To be hit on next release

• bug 628795 - en-US.xpi uploaded to the wrong place during 1.9.1/1.9.2 releases
• bug 616512 - Linux repacks will always fail to upload if run before win32 build finishes
• 2nd run of mac partner repacks fails because previous run did not remove mount point
  • TODO file

Sporadic

• bug 628390 - macosx_repack_4/6 failed because post_upload failed for 'ko'
• bug 628386 - win32_repack_5/6 failed to timeout on w32-ix-slave23 being slow
• bug 557268 - "update_verify" dependent scheduler did not fire
• macosx_repack_2/6 died with SIGUSR1
  • TODO check if it needs filing

Build data

Build #	Type	Build ID	Build machine	Time to build
1	Linux	20110121145245	mv-moz2-linux-ix-slave11	38 mins, 28 secs
	Mac	20110121145415	bm-xserve16	1 hrs, 39 mins, 47 secs
	Windows	20110121145327	w32-ix-slave23	4 hrs, 3 mins, 53 secs

Build 1

• Clobbered with "Any master/any release/any builder"
• tagged buildbotcustom, compare-locales and tools with FIREFOX_3_6_14_BUILD1 and FIREFOX_3_6_14_RELEASE
• landed config updates, including l10n-changesets from here
• tagged buildbot-configs on "production" branch with FIREFOX_3_6_14_BUILD1 and FIREFOX_3_6_14_RELEASE
• merge to production
• updated /tools/buildbotcustom/buildbotcustom, /builds/buildbot/configs, and /builds/buildbot/builder_master/tools to latest
• reconfigured master (Not needed - release_sanity.py takes care of it)
• Left tree open, 0.8.0 automation doesn't require that it closes
• kicked off automation with:

PYTHONPATH=/builds/buildbot/builder_master:/builds/buildbot/builder_master/tools/lib/python python /builds/buildbot/builder_master/tools/buildbot-helpers/release_sanity.py -u armenzg -V 3.6.14 --branch mozilla-1.9.2 --build-number 1 -c release-firefox-mozilla-1.9.2.py --dryrun localhost:9010

Tag

No problems.

Version bump

Don't forget to land the version bumps after tagging completes.

By the time we do the next release we might have bug 607372 fixed.

Source / XULRunner Source

No problems.

Build

• No problems

Remove en-US.xpi

You want to remove the en-US.xpi before autosigning creates the MD5SUMS and the SHA1SUMS.

TODO: check to see which bug to link here to know that the problem is fixed.

     One of these bugs should do: bug 609878 or bug 485860. 

Please remove this section once this problem is fixed.

Start autosigning

If you are expecting to sign during your workday you can start auto signing to take care of the deliverables as they are uploaded to FTP.

L10n Repack

• macosx_repack_2/6 died with SIGUSR1 before attempting any repacks
  • Manually rebooted bm-xserve16
  • Retriggered the builder with "rebuild"
  • check bugs hit section for any filed bug
• macosx_repack_4/6 failed because post_upload failed for 'ko'
  • used "macosx_standalone_repack" to re-trigger 'ko' as documented in here
  • check bugs hit section for any filed bug
• win32_repack_5/6 failed to timeout on w32-ix-slave23 being slow
  • gracefully shutdown the slave and re-triggered the job
  • check bugs hit section for any filed bug

Partner Repack

• Linux & Mac partner repack failed to upload builds bug 616512
  • the way to recover until the bug is fixed is to wait for win32 build to finish and then just re-trigger; no clobber is needed as it is not used by the builder
  • instead I failed to read note from bhearsum and I did the following
    • tried to debug locally on "bm-xserve22" why it did not work but when run it worked

cd /builds/slave/rel-192-lnx-partner-rpk/./partner-repacks/scripts/repacked_builds/3.6.14/build1
rsync -av -e ssh -oIdentityFile=~/.ssh/ffxbld_dsa . \
ffxbld@stage-old.mozilla.org:/pub/mozilla.org/firefox/nightly/3.6.14-candidates/build1/unsigned/partner-repacks
bash -c 'ssh -oIdentityFile=~/.ssh/ffxbld_dsa ffxbld@stage-old.mozilla.org \
touch /pub/mozilla.org/firefox/nightly/3.6.14-candidates/build1//unsigned/partner-repacks/partner_build_linux'
# after touching the autosign system can proceeed

• 2nd resubmitted Mac repack failed
  • the reason is that the first job that failed did not run the cleanup job
  • check bugs hit section for any filed bug

• It is all fine now

XULRunner Build

All good.

Unittests

Linux:

• hit bug 565437 timeout which aborted the run; I re-triggered it

Darwin:

• hit bug 501221 or {bug|502646}} timeout; nothing to be done

Windows:

• green

Signing

Followed CombinedSigning doc.

I did remove the en-US.xpi file too late.

• bug filed under the bugs hit section

To recover I did the following:

#ffxbld@stage-old.mozilla.org
rm /home/ftp/pub/mozilla.org/firefox/nightly/3.6.14-candidates/build1/en-US.xpi

On the keymaster:

cd ~/signing-work/firefox-3.6.4/signed_build1
rm en-US.xpi MD5SUMS* SHA1SUMS*
# I run the following command but by running each individually
# because I didn't know I could run it in one shot
# NOTE that "stage", "verify" and "fake-upload" are targets 
# that happen after "sign2" is run
# http://hg.mozilla.org/build/tools/file/tip/release/signing/Makefile#l73
make checksum-files create-sigs verify-sigs quick-verify stage verify fake-upload PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} RELCONFIG=${RELCONFIG} USE_NEW=1

Then continued by uploading as it would normally be done.

L10n Verify

Many string changes across many platforms/locales. All seemed legitimate, didn't dig further.

Updates

• completion on Monday Jan. 24th @ 15:30pm PST.
• QA signed off @ 4:22pm PST

3.6* -> 3.6.13

Generation

Got triggered after signing bits appeared on ftp.

No problems.

Update Verify

I had to "force build" the "update_verify" since reconfigures happened since the release was triggered. (added to bugs hit section)

All went green.

Socorro product data updates

Added '3.6.14' to crash-stats with this admin page.

TODO add "3.6.15pre". I have to land version bump first.

Update Bouncer

Used "Force Build" to start the "release-mozilla-1.9.2-bouncer_submitter" builder.

All good.