Releases/Firefox 4.0b7/BuildNotes
Notes About Releasing
Please update the Release:Primer for future releases (bug fixes, changes to automation) as needed
Build Engineers
armenzg - bug 596259
Signed-off Revision(s)
- Build 1: f7016571b472
L10N changesets
- checked with Pike about two pending reviews (not taken)
- wget https://l10n-stage-sj.mozilla.org/shipping/l10n-changesets?ms=fx4.0b7
- hit "ship it" from FF4 beta7 page
Tags
Build # | Tag | Changeset |
1 | GECKO20b7_20101104_RELBRANCH | f7016571b472 |
FIREFOX_4_0b7_BUILD1/FIREFOX_4_0b7_RELEASE | 297086a0fb61 |
Build data
Build # | Type | Build ID | Build machine | Time to build |
1 | Linux [compressed .tar.bz2] | 20101104125541 | mv-moz2-linux-ix-slave17 | 55 mins, 38 secs |
Linux64 [compressed .tar.bz2] | 20101104131204 | moz2-linux64-slave09 | 1hr 35 mins | |
Macosx64 [.dmg] | 20101104131842 | moz2-darwin10-slave19 | 4hrs 26mins | |
Win32 [.exe/.zip] | 20101104142426 | mw32-ix-slave03 | 2hrs 46mins |
Using pm:8010.
Bugs hit
During this release we have hit the following bugs:
- [NEW] bug 589914 - tag step should not fail on APPROVAL REQUIRED
- [BLOCKED ON SCHEDULERDB] bug 553300 - we have to move 10.5 slaves to run l10n verification. The day we share pools with production we won't hit this. I added a note on the wiki note to remember to bring them over for the next release.
- [BLOCKED ON SCHEDULERDB] bug 609887 - we currently run repacks in a subset of slaves. I moved slaves that were not in the subset for needed for macosx64_repack. I added a note on the wiki note to remember to bring them over for the next release.
- [FIXED] bug 609932 - L10n Verification failed for Linux because we now have checksums
- [ENHANCEMENT] bug 609981 - Create a TBPL page for Firefox-Release Tinderbox page. This will ease finding release unit test and performance test jobs going orange.
Build 1
Setup
- marked "any master" for "any release" for "any builder" to clobber (not connected to Build-VPN to force LDAP auth - https://build.mozilla.org/clobberer - check the date for one of release builders to confirm it worked)
- moving slaves
- from pm01:
- mv-moz2-linux-ix-slave{08,10,17}, linux-ix-slave13
bm-xserve{15,18,19,21}We don't have 10.5 for this beta- moz2-darwin10-slave{18,19,40-43,45-46}
- mw32-ix-slave{03,04,09,10,12}
- from pm03:
- moz2-linux64-slave0{1,2,8,9,12}
- from pm01:
- moved more slaves (see "Bugs hit" section - I did this after build jobs started)
- from pm01:
- moz2-darwin9-slave{05,06,07,09,11,12} - so we can do L10n verification
- moz2-darwin10-slave{06,07,08,09,11,12} - so we can do L10n Macos64 repacks
- from pm01:
- received GO
- closed TREE
- submitted patch for review with new l10n-changeset and driver's changeset
- Tagged buildbot-configs, buildbotcustom and tools
# buildbot-configs hg tag -r 23deb7d0556b FIREFOX_4_0b7_BUILD1 hg tag -r 23deb7d0556b FIREFOX_4_0b7_RELEASE # tools hg tag -r cd019fbfb429 FIREFOX_4_0b7_BUILD1 hg tag -r cd019fbfb429 FIREFOX_4_0b7_RELEASE #buildbotcustom hg tag -r ed7c08dbed63 FIREFOX_4_0b7_BUILD1 hg tag -r ed7c08dbed63 FIREFOX_4_0b7_RELEASE
- Landed config changes
- removed *.pyc from moz2-master dir in pm:8010
- reconfig'd
- Trigger the automation
buildbot sendchange --username=armenzg --master=localhost:9010 --branch=mozilla-central -m "Firefox 4.0b7 build1" Go
Tag
slave: linux-ix-slave13
- tagging failed on the last step because I used HTML comment on the tinderbox admin page. Failed with:
hg push -e "ssh -l ffxbld -i ~cltbld/.ssh/ffxbld_dsa" -f ssh://hg.mozilla.org/mozilla-central ... pushing to ssh://hg.mozilla.org/mozilla-central searching for changes remote: adding changesets remote: adding manifests remote: adding file changes remote: added 3 changesets with 5 changes to 4 files remote: Pushing to an APPROVAL REQUIRED tree requires your top changeset comment to include: a=... (or, more accurately, a\S*=...) remote: transaction abort! remote: rollback completed remote: abort: pretxnchangegroup.a_treeclosure hook failed abort: unexpected response: empty string
- I should have remove the following instead of commenting it out:
<!--<span id="treestatus" style="color: red;">APPROVAL REQUIRED</span> for Gecko 2.0b7 blockers or Fennec 2.0b2 blockers - see the <a href="https://wiki.mozilla.org/Tree_Rules#mozilla-central_-_Trunk_.28Firefox_4.0.2C_Gecko_2.0_work.29">Tree Rules</a>. )-->
- After fixing tree rule I logged into linux-ix-slave13 and run the step manually:
cd /builds/slave/tag/mozilla-central hg push -e "ssh -l ffxbld -i ~cltbld/.ssh/ffxbld_dsa" -f ssh://hg.mozilla.org/mozilla-central
- reconfigured master with Dummy factory and did another sendchange
- Re-opened the tree
- Filed bug 609717
Source
slave: linux-ix-slave13
- No problems
Build/Repack
- Linux: green, no problems
- Linux64: green, no problems
- Mac64: green, no problems (took too long)
- Win32: green, no problems
Unittests
Results
TODO review each one later and verify they are known.
Looked at Firefox-Release tbox page to find orange jobs.
Oranges:
- [Rev3 Fedora 12 release-mozilla-central talos dromaeo http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Release/1288905204.1288908545.9308.gz]
- Rev3 MacOSX Leopard 10.5.8 release-mozilla-central opt test mochitest-other
- Rev3 Fedora 12 release-mozilla-central opt test mochitest-other
Signing
- No en-US.xpi to remove from stage, bug 485860 is FIXED. May need reopening as no en-US.xpi in $platform/xpi either
- Used instructions
- Hit an error on the verify-asc target. Filed bug 609816. Fallout from the three new files
linux-i686/en-US/firefox-4.0b7.checksums
- ran the fake-upload target manually, then upload
L10N Verify
- Linux L10n and Linux L64 verification failed with:
Unknown package type for file: firefox-4.0b7-build1/linux-i686/en-US/firefox-4.0b7.checksums Unknown package type for file: firefox-4.0b7-build1/linux-x86_64/en-US/firefox-4.0b7.checksums
- bug 609932 - L10n Verification failed for Linux because we now have checksums
- land fix
- clobber
- force build both builders that failed
- All green
Updates
No problems
Special case for old mac universal builds
In this beta we are not generating the old ppc+386 universal builds for mac. Therefore we have to play a little bit with snippets to make them update to the new 32-bit and 64-bit updates.
To do so we are doing "Darwin_Universal-gcc3" and "Darwin_Universal-gcc3" directories with the contents of "Darwin_x86_64-gcc3" which point to beta7. We are also not giving partial updates from the old universal to the new universal since we don't have a partial mar for that case and it was not easy to generate.
Here is an overview of how we did it and in the following section the mkdir & rsync dance that we have to do.
#cltbld@aus-staging cd /opt/aus2/snippets/staging/ cp -r 20101104-Firefox-4.0b7-test 20101104-Firefox-4.0b7-test-fixed cd 20101104-Firefox-4.0b7-test-fixed/Firefox # mkdir & rsync dance for each 3.7* and 4.0* directory ~/bin/pushsnip 20101104-Firefox-4.0b7-test-fixed cd - cp -r 20101104-Firefox-4.0b7 20101104-Firefox-4.0b7-fixed cd 20101104-Firefox-4.0b7-fixed/Firefox # mkdir & rsync dance for each 3.7* and 4.0* directory # we push the fixed snippets in a latter section # after QA & rel-driver grants us to do it.
The mkdir & rsync dance mentioned in the previous blockquote:
mkdir -p 3.7a1/Darwin_Universal-gcc3/20100208064157 rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a1/Darwin_Universal-gcc3/20100208064157 mkdir -p 3.7a2/Darwin_Universal-gcc3/20100228155422 rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a2/Darwin_Universal-gcc3/20100228155422/ mkdir -p 3.7a3/Darwin_Universal-gcc3/20100315065900 rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a3/Darwin_Universal-gcc3/20100315065900/ mkdir -p 3.7a4/Darwin_Universal-gcc3/20100407105356 rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a4/Darwin_Universal-gcc3/20100407105356/ mkdir -p 3.7a5/Darwin_Universal-gcc3/20100610053455 rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a5/Darwin_Universal-gcc3/20100610053455 mkdir -p 4.0b1/Darwin_Universal-gcc3/20100630131607 rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 4.0b1/Darwin_Universal-gcc3/20100630131607/ mkdir -p 4.0b2/Darwin_Universal-gcc3/20100720175703 rsync -av --exclude partial.txt 4.0b2/Darwin_x86_64-gcc3/20100720180051/ 4.0b2/Darwin_Universal-gcc3/20100720175703/ mkdir -p 4.0b3/Darwin_Universal-gcc3/20100805181943/ rsync -av --exclude partial.txt 4.0b3/Darwin_x86_64-gcc3/20100805182338/ 4.0b3/Darwin_Universal-gcc3/20100805181943/ mkdir -p 4.0b4/Darwin_Universal-gcc3/20100818121614 rsync -av --exclude partial.txt 4.0b4/Darwin_x86_64-gcc3/20100818121922/ 4.0b4/Darwin_Universal-gcc3/20100818121614/ mkdir -p 4.0b5/Darwin_x86-gcc3-u-ppc-i386/20100831070010 rsync -av --exclude partial.txt 4.0b5/Darwin_x86_64-gcc3/20100831070403/ 4.0b5/Darwin_x86-gcc3-u-ppc-i386/20100831070010/ mkdir -p 4.0b6/Darwin_x86-gcc3-u-ppc-i386/20100914072643 rsync -av --exclude partial.txt 4.0b6/Darwin_x86_64-gcc3/20100914073111/ 4.0b6/Darwin_x86-gcc3-u-ppc-i386/20100914072643/
Update Verify
All green but I can see the following:
# for linux remove failed: -1,2 (searchplugins/bing.xml.moz-backup) # for osx remove failed: -1,2 (Contents/MacOS/searchplugins/bing.xml.moz-backup) remove failed: -1,2 (force_plist_reload.moz-backup) # for win32 remove failed: -1,2 (mozjs.dll.moz-backup) remove failed: -1,2 (searchplugins/bing.xml.moz-backup)
- bing.xml is new for all platforms.
- bhearsum mentions that force_plist_reload is known to be new
- khuey mentions that mozjs.dll is new on Windows for beta7
Return slaves
Put back
- moz2-linux64-slave{01,02,08,09,12}, since that's half our linux64 build pool
WE SHOULD MOVE SOME WIN32/LINUX IX BOXES BACK TOO.
Final checks before push
Virus check
Update Bouncer
Push to mirrors
- Wait for email from driver before continuing
Final verify
Push snippets to beta channel
Remove index.html files
Release