Releases/Firefox 3.6.14/BuildNotes: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(Adding build 2 section)
Line 24: Line 24:
* 2nd run of mac partner repacks fails because previous run did not remove mount point
* 2nd run of mac partner repacks fails because previous run did not remove mount point
** TODO file bug
** TODO file bug
* TODO file bug for the need to run this command:
find /pub/mozilla.org/firefox/nightly/3.6.14-candidates/build1/unsigned/partner-repacks -exec chmod g-w "{}" \;


== Sporadic ==
== Sporadic ==

Revision as of 19:02, 7 February 2011

Build Engineers

armenzg - bug 624496

Signed-off Revision(s)

28ce03bbd37c

Tags

Build # Tag Changeset
1 FIREFOX_3_6_14_BUILD1 FIREFOX_3_6_14_RELEASE http://hg.mozilla.org/releases/mozilla-1.9.2/rev/69bfc730b3e9

Bugs hit

To be hit on next release unless fixed

You won't hit these bugs if they are fixed first:

  • bug 628795 - en-US.xpi uploaded to the wrong place during 1.9.1/1.9.2 releases
  • bug 616512 - Linux repacks will always fail to upload if run before win32 build finishes
  • bug 629256 - Fix snippets for some locales
  • 2nd run of mac partner repacks fails because previous run did not remove mount point
    • TODO file bug
  • TODO file bug for the need to run this command:
find /pub/mozilla.org/firefox/nightly/3.6.14-candidates/build1/unsigned/partner-repacks -exec chmod g-w "{}" \;

Sporadic

  • bug 628390 - macosx_repack_4/6 failed because post_upload failed for 'ko'
  • bug 628386 - win32_repack_5/6 failed to timeout on w32-ix-slave23 being slow
  • bug 557268 - "update_verify" dependent scheduler did not fire
  • macosx_repack_2/6 died with SIGUSR1
    • TODO check if it needs filing

Build data

Build # Type Build ID Build machine Time to build
1 Linux 20110121145245 mv-moz2-linux-ix-slave11 38 mins, 28 secs
Mac 20110121145415 bm-xserve16 1 hrs, 39 mins, 47 secs
Windows 20110121145327 w32-ix-slave23 4 hrs, 3 mins, 53 secs

Build 1

  • Clobbered with "Any master/any release/any builder"
  • tagged buildbotcustom, compare-locales and tools with FIREFOX_3_6_14_BUILD1 and FIREFOX_3_6_14_RELEASE
  • landed config updates, including l10n-changesets from here
  • tagged buildbot-configs on "production" branch with FIREFOX_3_6_14_BUILD1 and FIREFOX_3_6_14_RELEASE
  • merge to production
  • updated /tools/buildbotcustom/buildbotcustom, /builds/buildbot/configs, and /builds/buildbot/builder_master/tools to latest
  • reconfigured master (Not needed - release_sanity.py takes care of it)
  • Left tree open, 0.8.0 automation doesn't require that it closes
  • kicked off automation with:
PYTHONPATH=/builds/buildbot/builder_master:/builds/buildbot/builder_master/tools/lib/python python /builds/buildbot/builder_master/tools/buildbot-helpers/release_sanity.py -u armenzg -V 3.6.14 --branch mozilla-1.9.2 --build-number 1 -c release-firefox-mozilla-1.9.2.py --dryrun localhost:9010

Tag

No problems.

Version bump

Please verify on your next release that bug 607372 is fixed.

Once you do VERIFY the bug and remove this section.

Update Bouncer

Used "Force Build" to start the "release-mozilla-1.9.2-bouncer_submitter" builder.

All good.

Stage EU Ballot Builds

# ffxbld@stage
cd /home/ftp/pub/firefox/nightly/3.6.14-candidates/build1
mv -v partner-repacks/euballot/win32 win32-EUballot

Please remove this section once bug 614227 is FIXED.

Source / XULRunner Source

No problems.

Build

  • No problems

Remove en-US.xpi

You want to remove the en-US.xpi before autosigning creates the MD5SUMS and the SHA1SUMS.

Please remove this section once bug 628795 is fixed.

Check here if you don't do remove en-US.xpi on time and want to know how to recover.

Start autosigning

If you are expecting to sign during your workday you can start auto signing to take care of the deliverables as they are uploaded to FTP.

L10n Repack

  • macosx_repack_2/6 died with SIGUSR1 before attempting any repacks
    • Manually rebooted bm-xserve16
    • Retriggered the builder with "rebuild"
    • check bugs hit section for any filed bug
  • macosx_repack_4/6 failed because post_upload failed for 'ko'
    • used "macosx_standalone_repack" to re-trigger 'ko' as documented in here
    • check bugs hit section for any filed bug
  • win32_repack_5/6 failed to timeout on w32-ix-slave23 being slow
    • gracefully shutdown the slave and re-triggered the job
    • check bugs hit section for any filed bug

Partner Repack

  • Linux & Mac partner repack failed to upload builds bug 616512
    • the way to recover until the bug is fixed is to wait for win32 build to finish and then just re-trigger; no clobber is needed as it is not used by the builder
    • instead I failed to read note from bhearsum and I did the following
      • tried to debug locally on "bm-xserve22" why it did not work but when run it worked
cd /builds/slave/rel-192-lnx-partner-rpk/./partner-repacks/scripts/repacked_builds/3.6.14/build1
rsync -av -e ssh -oIdentityFile=~/.ssh/ffxbld_dsa . \
ffxbld@stage-old.mozilla.org:/pub/mozilla.org/firefox/nightly/3.6.14-candidates/build1/unsigned/partner-repacks
bash -c 'ssh -oIdentityFile=~/.ssh/ffxbld_dsa ffxbld@stage-old.mozilla.org \
touch /pub/mozilla.org/firefox/nightly/3.6.14-candidates/build1//unsigned/partner-repacks/partner_build_linux'
# after touching the autosign system can proceeed
  • 2nd resubmitted Mac repack failed
    • the reason is that the first job that failed did not run the cleanup job
    • check bugs hit section for any filed bug
  • It is all fine now

XULRunner Build

All good.

Unittests

Linux:

  • hit bug 565437 timeout which aborted the run; I re-triggered it

Darwin:

Windows:

  • green

Signing

Followed CombinedSigning doc.

I did remove the en-US.xpi file too late.

To recover I did the following: 

#ffxbld@stage-old.mozilla.org
rm /home/ftp/pub/mozilla.org/firefox/nightly/3.6.14-candidates/build1/en-US.xpi

On the keymaster: 

cd ~/signing-work/firefox-3.6.4/signed_build1
rm en-US.xpi MD5SUMS* SHA1SUMS*
# I run the following command but by running each individually
# because I didn't know I could run it in one shot
# NOTE that "stage", "verify" and "fake-upload" are targets 
# that happen after "sign2" is run
# http://hg.mozilla.org/build/tools/file/tip/release/signing/Makefile#l73
make checksum-files create-sigs verify-sigs quick-verify stage verify fake-upload PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} RELCONFIG=${RELCONFIG} USE_NEW=1

Then continued by uploading as it would normally be done.

L10n Verify

Many string changes across many platforms/locales. All seemed legitimate, didn't dig further.

Updates

  • completion on Monday Jan. 24th @ 15:30pm PST.
  • QA signed off @ 4:22pm PST

3.6* -> 3.6.13

Generation

Got triggered after signing bits appeared on ftp.

No problems.

Update Verify

I had to "force build" the "update_verify" since reconfigures happened since the release was triggered. (added to bugs hit section)

All went green.

Socorro product data updates

Added '3.6.14' & '3.6.15pre' to crash-stats with this admin page.

Going to beta day!!

Do this in the morning before the rel-driver gives you the GO. 

#From a build slave you can reach aus2-staging
#cltbld@aus2-staging.mozilla.org
~/bin/backupsnip 20110124-Firefox-3.6.14-beta #takes 20-50mins - depends if other releases are backing up

Push to Beta Channel

#From a build slave you can reach aus2-staging
#cltbld@aus2-staging.mozilla.org
#This takes now a lot of time because of PHX datacenter
~/bin/pushsnip 20110124-Firefox-3.6.14-beta

Fix snippets for some locales

See bug 629256 

cd /opt/aus2/snippets/staging

rsync -a 20110124-Firefox-3.6.14/ 20110126-Firefox-3.6.14-bug629256fix/
touch 20110126-Firefox-3.6.14-bug629256fix/
rm -rf 20110126-Firefox-3.6.14-bug629256fix/Firefox/3.6*/Darwin*/*/{ml,te}

rsync -a 20110124-Firefox-3.5.17-3.6.14-MU/
20110126-Firefox-3.5.17-3.6.14-MU-bug629256fix/
touch 20110126-Firefox-3.5.17-3.6.14-MU-bug629256fix/
rm -rf
20110126-Firefox-3.5.17-3.6.14-MU-bug629256fix/Firefox/3.5*/Darwin*/*/{kn,ml,te}

Going live day!

Final checks before push & virus scan

Go to the "pre push checks" builder and add these two values:

  • Revision to build FIREFOX_3_6_14_RELEASE
  • Property 1 name: release_config: mozilla/release-firefox-mozilla-1.9.2.py

Push to Mirrors

DONE TO HERE

Go to the "push to mirros" builder and add these two values:

  • Revision to build FIREFOX_3_6_14_RELEASE
  • Property 1 name: release_config: mozilla/release-firefox-mozilla-1.9.2.py

Final Verification

Once mirror uptake has reached 50 or 60% this builder can be started with 'Force Build' from the Buildbot waterfal

Push release channel snippets

Please use the 20110126-Firefox-3.6.14-bug629256fix directory when pushing to the release channel for 3.6.x -> 3.6.14 updates, for bug 629256. We drop updates for a few locales which have strings that make the updater crash. NThomas 23:52, 26 January 2011 (PST)

Push 3.0.19 -> 3.6.13 MU snippets

Push EU Ballot build

  • Filed bug 632052 to change download redirect
  • On bouncer we add "Firefox-3.6.14-EUballot" product under "Mirrors->Products->Add new product"
  • Once added, we add a new mirror location:
/firefox/releases/3.6.14/win32-EUballot/:lang/Firefox%20Setup%203.6.14.exe

  • Check with Tomcat to see if he has filed the website bug

Release

XULRunner Signing

Push XULRunner to mirrors

Update XULRunner entries on wiki

Build 2

Tag

Version bump

Update Bouncer

Source / XULRunner Source

Build

Start autosigning

If you are expecting to sign during your workday you can start auto signing to take care of the deliverables as they are uploaded to FTP.

L10n Repack

Partner Repack

XULRunner Build

Unittests

Signing

L10n Verify

Updates

3.6* -> 3.6.13

Generation
Update Verify

Socorro product data updates

Already done for build 1

Going to beta day!!

Push to Beta Channel

Fix snippets for some locales

Going live day!

Final checks before push & virus scan

Go to the "pre push checks" builder and add these two values:

  • Revision to build FIREFOX_3_6_14_RELEASE
  • Property 1 name: release_config: mozilla/release-firefox-mozilla-1.9.2.py

Push to Mirrors

Go to the "push to mirros" builder and add these two values:

  • Revision to build FIREFOX_3_6_14_RELEASE
  • Property 1 name: release_config: mozilla/release-firefox-mozilla-1.9.2.py

Final Verification

Once mirror uptake has reached 50 or 60% this builder can be started with 'Force Build' from the Buildbot waterfal

Push release channel snippets

Please use the 20110126-Firefox-3.6.14-bug629256fix directory when pushing to the release channel for 3.6.x -> 3.6.14 updates, for bug 629256. We drop updates for a few locales which have strings that make the updater crash. NThomas 23:52, 26 January 2011 (PST)

Push 3.0.19 -> 3.6.13 MU snippets

Push EU Ballot build

  • Filed bug 632052 to change download redirect
  • On bouncer we add "Firefox-3.6.14-EUballot" product under "Mirrors->Products->Add new product"
  • Once added, we add a new mirror location:
/firefox/releases/3.6.14/win32-EUballot/:lang/Firefox%20Setup%203.6.14.exe

  • Check with Tomcat to see if he has filed the website bug

Release

XULRunner Signing

Push XULRunner to mirrors

Update XULRunner entries on wiki

Retrieved from "https://wiki.allizom.org/index.php?title=Releases/Firefox_3.6.14/BuildNotes&oldid=282890"