Releases/Firefox 14.0.1/BuildNotes: Difference between revisions
| Line 194: | Line 194: | ||
* renaming Firefox-14.0.1-Partial-13.0.2 to Firefox-14.0.1-Partial-13.0.1 | * renaming Firefox-14.0.1-Partial-13.0.2 to Firefox-14.0.1-Partial-13.0.1 | ||
* s/13.0.2/13.0.1/ in the 8 locations for that product | * s/13.0.2/13.0.1/ in the 8 locations for that product | ||
Added Firefox-14.0.1-Partial-12.0 product and location for win, for that special partial. | |||
Additional work still to be done for funnelcake, see [https://bugzilla.mozilla.org/show_bug.cgi?id=758795#c8 notes]. Yes, it's a bit strange to put 14.0.1 updates in 13.0.1 dirs, but we already have a set up to block CDN/mirrors as needed. | Additional work still to be done for funnelcake, see [https://bugzilla.mozilla.org/show_bug.cgi?id=758795#c8 notes]. Yes, it's a bit strange to put 14.0.1 updates in 13.0.1 dirs, but we already have a set up to block CDN/mirrors as needed. | ||
Revision as of 02:28, 17 July 2012
Notes About Releasing
Please update the Notes Template and the Release:Primer for future releases (bug fixes, changes to automation) as needed
Bugs hit
Enter any bugs pre-existing or newly discovered and filed during the release:
- bug 773876 - partner repack's download_pkg-dmg step does not retry failed wget
Build Engineers
nthomas with a start from bhearsum - Tracking bug: bug 758795
Signed-off Revision(s)
Build 1: 228ba1a111fc
L10N changesets
Instructions on how to get them
- Firefox: https://l10n-stage-sj.mozilla.org/shipping/l10n-changesets?ms=fx14.0.1
- Fennec: https://l10n-stage-sj.mozilla.org/shipping/json-changesets?ms=fennec14.0.1&platforms=android&multi_android-multilocale_repo=releases%2Fmozilla-beta&multi_android-multilocale_rev=default&multi_android-multilocale_path=mobile%2Fandroid%2Flocales%2Fmaemo-locales
Tags
Manually tag the automation code, then record the generated tags below. (details)
| Build # | Branch, Tags | Changeset |
| 1 | MOBILE1401_2012071313_RELBRANCH, FENNEC_14_0_1_RELEASE, FENNEC_14_0_1_BUILD1 | 9c64e364f5df |
| GECKO1401_2012071313_RELBRANCH, FIREFOX_14_0_1_BUILD1 FIREFOX_14_0_1_RELEASE | e5728a4e106c | |
| 2 | MOBILE1401_2012071316_RELBRANCH FENNEC_14_0_1_BUILD2 FENNEC_14_0_1_RELEASE | 5ca6bd1f2bdf |
Build data
Firefox
| Build # | Type | Build ID | Build machine | Time to build |
| 1 | Linux | 20120713134347 | inux-ix-slave39 | 1 hrs, 9 mins, 10 secs |
| Linux64 | linux64-ix-slave16 | 1 hrs, 4 mins, 38 secs | ||
| Mac | bld-lion-r5-063 | 1 hrs, 19 mins, 34 secs | ||
| Windows | mw32-ix-slave06 | 3 hrs, 41 mins, 16 secs |
Fennec
| Build # | Type | Build ID | Build machine | Time to build |
| 1 | android | 20120713133725 | linux-ix-slave06 | 50 mins, 41 secs |
| 2 | android | 20120713170118 | linux-ix-slave39 | 52 mins, 49 secs |
Notes
Build 1
Diff mozconfigs
From a mozilla-release checkout (with 13.0.1 content), ran:
for p in linux32 linux64 macosx-universal win32; do diff -Naur {,../mozilla-beta/}browser/config/mozconfigs/$p/release; done
(note: mozilla-beta had not been updated to 15.0 code yet.)
No differences, nothing to do here.
Set reserved slaves
- set reserved_slaves to 8:
cd /builds/buildbot/build1 echo 8 > master/reserved_slaves
Preparing to start Automation
- set a clobber for any master, release-mozilla-release, any builder as a speed up
- create a symlink to the candidates directory
# ffxbld or tbirdbld@stage version=14.0.1 product=firefox cd /pub/mozilla.org/$product/nightly/ mkdir ../candidates/$version-candidates ln -s ../candidates/$version-candidates $version-candidates
- moved the slavealloc lock for 6 mw32-ix-slave from bm13 to bm12, rebooted them. mw32-ix-slave04 needed TLC for fuzzing exhausting e drive (bug 771687). Plenty of linux/linux64 slaves already. Locked 8 bld-r5-lion slaves to bm12.
- Shipped the l10n milestones
- Clicked "Ship" for Fennec
- Clicked "Add"
- repo: releases/mozilla-release
- branch: default
- path: mobile/android/locales/maemo-locales
- Clicked "Ship it"
- Clicked "Add"
- Clicked "Ship" for Firefox
- Clicked "Ship it"
- Clicked "Ship" for Fennec
- Landed buildbot-configs patch w/ updated l10n-changesets:
wget -O- --no-check-certificate "https://bugzilla.mozilla.org/attachment.cgi?id=641885" | patch -p1 sed -i -e "s/TBD/'228ba1a111fc'/" mozilla/release-firefox-mozilla-release.py mozilla/release-fennec-mozilla-release.py wget -O mozilla/l10n-changesets_mobile-release.json "https://l10n-stage-sj.mozilla.org/shipping/json-changesets?ms=fennec14.0.1&platforms=android&multi_android-multilocale_repo=releases%2Fmozilla-beta&multi_android-multilocale_rev=default&multi_android-multilocale_path=mobile%2Fandroid%2Flocales%2Fmaemo-locales" wget -O mozilla/l10n-changesets_mozilla-release "https://l10n-stage-sj.mozilla.org/shipping/l10n-changesets?ms=fx14.0.1" # Remove the locales we aren't shipping from l10n changesets sed -i -e '/^mn.*$/d' -e '/^sw.*$/d' mozilla/l10n-changesets_mozilla-release hg commit -m "bug 758795: tracking bug for build and release of Firefox/Fennec 14.0.1. r=aki" hg up -r production hg transplant default hg tag -r production {FIREFOX,FENNEC}_14_0_1_{RELEASE,BUILD1} hg push ssh://hg.mozilla.org/build/buildbot-configs
- Tagged buildbotcustom:
hg tag -r production-0.8 {FIREFOX,FENNEC}_14_0_1_{RELEASE,BUILD1}
hg push ssh://hg.mozilla.org/build/buildbotcustom
- Tagged tools:
hg tag -r default {FIREFOX,FENNEC}_14_0_1_{RELEASE,BUILD1}
hg push ssh://hg.mozilla.org/build/tools
- Updated and reconfiged buildbot-master12:
cd /builds/buildbot/build1 source bin/activate make update reconfig
- Ran release sanity in dry run mode:
cd /builds/buildbot/build1/master source ../bin/activate PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u bhearsum \ -V 14.0.1 --branch mozilla-release --build-number 1 \ --release-config release-firefox-mozilla-release.py \ --release-config release-fennec-mozilla-release.py --products firefox,fennec \ --dryrun localhost:9001
- Complained about mn and sw being missing, because the l10n dashboard is wrong
- Ran release sanity with --bypass-l10n-check to get the release going:
PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u bhearsum \ -V 14.0.1 --branch mozilla-release --build-number 1 \ --release-config release-firefox-mozilla-release.py \ --release-config release-fennec-mozilla-release.py --products firefox,fennec \ --bypass-l10n-check localhost:9001
E-mail Metrics
Sent the following mail to metrics-alerts@mozilla.org:
Firefox,firefox,firefox,14.0.1,14.0 Fennec,firefox-mobile,fennec,14.0.1,14.0
E-mail mirrors@mozilla.org
Followed up to IT message about CDN usage.
Edit rsync exclude files
Committed this diff, after review:
Index: rsyncd-mozilla-releases.exclude =================================================================== --- rsyncd-mozilla-releases.exclude (revision 43094) +++ rsyncd-mozilla-releases.exclude (working copy) @@ -1,3 +1,12 @@ ++ firefox ++ firefox/releases ++ firefox/releases/14.0.1 ++ firefox/releases/14.0.1/*/ ++ firefox/releases/14.0.1/*/*/ ++ firefox/releases/14.0.1/*/*/*/ ++ firefox/releases/14.0.1/**index.html +- firefox/releases/14.0.1/** +- firefox/releases/13.0.1-funnelcake11/update - firefox/releases/0.10 - firefox/releases/0.10.1 - firefox/releases/0.10rc
Tag
No problems
Bouncer Submitter
Bogus 13.0.2 oldVersion (see Updates for diagnosis) resulted in
- renaming Firefox-14.0.1-Partial-13.0.2 to Firefox-14.0.1-Partial-13.0.1
- s/13.0.2/13.0.1/ in the 8 locations for that product
Added Firefox-14.0.1-Partial-12.0 product and location for win, for that special partial.
Additional work still to be done for funnelcake, see notes. Yes, it's a bit strange to put 14.0.1 updates in 13.0.1 dirs, but we already have a set up to block CDN/mirrors as needed.
Source
Failed because I removed the buildbot-configs mozconfig that this step depends on. Reverted with:
hg revert -r 6507 mozilla2/linux/mozilla-release/release/mozconfig
hg commit -m "Re-add linux mozconfig so source steps will work"
hg up -r production
hg transplant -r production
hg tag -f -r production {FIREFOX,FENNEC}_14_0_1_{RELEASE,BUILD1}
hg push ssh://hg.mozilla.org/build/buildbot-configs
Used "rebuild" to restart source steps.
Failed again with a configure error. Realized this was because I didn't merge the beta mozconfig -> production. Brought forward compiler changes in http://hg.mozilla.org/build/buildbot-configs/rev/466340b368b8 to fix this. Used rebuild to try again.
Failed again with the same error. I did some debugging on the machine and found that there were some CFLAGS/CXXFLAGS I didn't bring forward and did that in http://hg.mozilla.org/build/buildbot-configs/rev/b2b6a37879f2. Used rebuild to try again. Worked this time.
Build
Firefox
No problems
Firefox repacks
No problems
Fennec
No problems
Fennec repacks
No problems
XULRunner
Source
Failed with the same issue as Firefox source builder.
Build
- Mac build failed first time because of bug 753223. Rebuilt it, got success
- Other platforms no problems
Push to Mirrors
No problems
Partner Repack
Firefox
Mac jobs failed when it got a 500 error trying to d/l pkg-dmg from hg.m.o, bug 773876 filed. Success on rebuild. Other platforms fine.
Funnelcake builds are documented elsewhere.
Android Signing
FENNEC ONLY RESPIN DECLARED.
Checksums
No problems, but will need to rerun for 12.0 to 14.0.1 partials.
On the 2nd run, failed with:
mkdir: cannot create directory `/pub/mozilla.org/firefox/nightly/14.0.1-candidates/build1//contrib': File exists mkdir: cannot create directory `/pub/mozilla.org/firefox/nightly/14.0.1-candidates/build1//contrib-localized': File exists
That's OK though, because the SUMS files uploaded prior to that.
There are no checksums for the 13.0.1 partial MARs because they weren't generated on the build machines. I don't think we need to bother fixing that up (bhearsum).
I also noticed bug 774345 while verifying the new files, which had given us a bunch of 13.0 checksums in the original run. That's not a problem now because they were overwritten with 14.0.1-only sums files.
Updates
See Releases/Firefox_14.0.1/BuildNotes-Updates because it's too long to keep here.
Reset reserved slaves
# buildbot-master12 cd /builds/buildbot/build1/master/ echo 0 > reserved_slaves
Check permissions / AV scan
No problems, but will need to rerun after 12.0 - 14.0.1 partial generation. Re-run had no problems either.
Push to internal mirrors
Move away the 13.0.2-14.0.1 partials that we aren't going to use:
# ffxbld@stage
mkdir -p ~/14.0.1-candidates-partials/build1/update
rsync -av --include=update --include=update/* --include=update/*/* \
--include=update/*/*/firefox-13.0.2-14.0.1.partial.mar* --exclude=* \
/pub/mozilla.org/firefox/candidates/14.0.1-candidates/build1/ ~/14.0.1-candidates-partials/build1/
find /pub/mozilla.org/firefox/candidates/14.0.1-candidates/build1/ -type f \
-name 'firefox-13.0.2-14.0.1.partial.mar*' -exec rm -v {} \;
There is extra work to do here for funnelcake, see the bug
Final verification
Before Release Day
Run backupsnip for release snippets
cd /opt/aus2/snippets/staging ~/bin/backupsnip Firefox-14.0.1-build1
Push index file to mirrors
# ffxbld@stage version=14.0.1 cd /pub/mozilla.org/firefox/releases/$version wget --no-check-certificate -O index.html https://bugzilla.mozilla.org/attachment.cgi?id=631778 version=14 sed -i -e "s/13/$version/g" index.html for dir in `find . -mindepth 1 -type d `; do cp -pv index.html $dir/; done
Push to external mirrors
Unblock firefox/releases/13.0.1-funnelcake/update as well firefox/releases/14.0.1/.
Release Day
Update XULRunner wiki page
Check Throttling
Push to Release Channel (for beta releases and release releases)
We'll have extra snippets to push for funnelcake, and the 12.0-14.0.1 partial. The correct order is
- Firefox-14.0.1-build1-funnelcake11_12 for release-cck-mozilla11, release-cck-mozilla12
- Firefox-14.0.1-build1 for main snippets (including the what's new and 12.0 partial)
Update symlinks
Remove index.html files
Update browser choice website
Build 2 (Fennec only)
Preparing to start Automation
- Landed config update
- Retagged with FENNEC_14_0_1_{BUILD2,RELEASE}
- Updated, reconfiged.
- Ran release sanity, passed:
cd /builds/buildbot/build1 source bin/activae PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u bhearsum -V 14.0.1 --branch mozilla-release --build-number 2 --release-config release-fennec-mozilla-release.py --products fennec --dryrun localhost:9001
- Ran without dryrun to start:
PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u bhearsum -V 14.0.1 --branch mozilla-release --build-number 2 --release-config release-fennec-mozilla-release.py --products fennec localhost:9001
Tag
No problems
Source
No problems
Build
Fennec
No problems
Fennec repacks
No problems
Android Signing
# cltsign@signing1 cd ~/signing-work/mozharness hg pull -u && hg up -C -r default python2.6 ~/signing-work/mozharness/scripts/sign_android.py --config-file signing/android_mozilla-release.py --platform android
Release Day
Publish Fennec to the Play Store (for beta releases and release releases)
Push the files (for beta releases and release releases)
Play Store (for beta releases and release releases)
https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Play_Store