Releases/Firefox 14.0.1/BuildNotes: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
 
(30 intermediate revisions by 2 users not shown)
Line 5: Line 5:
Enter any bugs pre-existing or newly discovered and filed during the release:
Enter any bugs pre-existing or newly discovered and filed during the release:
* {{bug|773876}} - partner repack's download_pkg-dmg step does not retry failed wget
* {{bug|773876}} - partner repack's download_pkg-dmg step does not retry failed wget
* {{bug|774601}} - euballot builds put in wrong part of candidates directory


=Build Engineers=
=Build Engineers=
Line 163: Line 164:


=== E-mail mirrors@mozilla.org ===
=== E-mail mirrors@mozilla.org ===
nthomas will do this later
Followed up to IT message about CDN usage.


=== Edit rsync exclude files ===
=== Edit rsync exclude files ===
Line 195: Line 196:
* s/13.0.2/13.0.1/ in the 8 locations for that product
* s/13.0.2/13.0.1/ in the 8 locations for that product


Additional work still to be done for funnelcake, see [https://bugzilla.mozilla.org/show_bug.cgi?id=758795#c8 notes]. Yes, it's a bit strange to put 14.0.1 updates in 13.0.1 dirs, but we already have a set up to block CDN/mirrors as needed.
Added Firefox-14.0.1-Partial-12.0 product and location for win, for that special partial.
 
Funnelcake11:
* added products Firefox-14.0.1-Partial-13.0.1-Funnelcake11 & Firefox-14.0.1-Complete-Funnelcake11
* added win and osx locations for both
 
Funnelcake12:
* added products Firefox-14.0.1-Partial-13.0.1-Funnelcake12 & Firefox-14.0.1-Complete-Funnelcake12
* added win and osx locations for both
(both of these should be one-off for this release)


=== Source ===
=== Source ===
Line 237: Line 247:
=== Partner Repack ===
=== Partner Repack ===
==== Firefox ====
==== Firefox ====
Mac jobs failed when it got a 500 error trying to d/l pkg-dmg from hg.m.o, {{bug|773876}} filed. Success on rebuild. Other platforms fine.
Mac jobs failed when it got a 500 error trying to d/l pkg-dmg from hg.m.o, {{bug|773876}} filed. Success on rebuild. EUballot builds went to wrong place - {{bug|774601}}. Otherwise fine.  


TODO Come back and generate funnelcake13/14 on win32/mac.
Funnelcake builds are [https://bugzilla.mozilla.org/show_bug.cgi?id=773531 documented elsewhere].


=== Android Signing ===
=== Android Signing ===
Line 247: Line 257:
No problems, but will need to rerun for 12.0 to 14.0.1 partials.
No problems, but will need to rerun for 12.0 to 14.0.1 partials.


=== Updates ===
On the 2nd run, failed with:
First problem: Failed to bump the patcher config
perl ../tools/release/patcher-config-bump.pl -p firefox -r Firefox -v 14.0.1 -a 14.0.1 \
-o 13.0.2 -b 1 ....
ASSERT: BumpFilePath() - Could not bump path: 'firefox/nightly/13.0.1-candidates/build1/update/%platform%/%locale%/firefox-13.0-13.0.1.partial.mar' from '13.0.2' to '14.0.1' at ../tools/lib/perl/Release/Patcher/Config.pm line 148.
due to incorrect oldVersion in release config.
 
Fix:
# buildbot-configs @ ed8561aa1056
sed -e 's/13_0_2/13_0_1/' -e 's/13\.0\.2/13.0.1/' -i '' mozilla/release-firefox-mozilla-release.py
hg ci -m "Bug 758795, fix up oldVersion for FF 14.0.1, r=version-numbers-are-confusing"
hg up -r production
hg transplant 1aa1d4209bb1
hg tag -f -r production {FIREFOX,FENNEC}_14_0_1_{RELEASE,BUILD1}
hg push ssh://hg.mozilla.org/build/buildbot-configs
 
# bm12
cd /builds/buildbot/build1
source bin/activate
make update reconfig
 
Note that the tagging was wrong for Fennec, revert it
hg tag -f -r b52a7634bca669ea1205ad34ab297b845ec193c1 FENNEC_14_0_1_RELEASE
hg tag -f -r b2b6a37879f23438c3fb29124f9b3d37e61cff13 FENNEC_14_0_1_BUILD1
# after verification using hg serve
hg push ssh://hg.mozilla.org/build/buildbot-configs
 
Forced builder.
 
Second problem:
Failed to upload aus2-test because in the create_snippets step we got
Partial patch info - 0 to create
Complete patch info - 0 to create
Past release patch info - 6168 to create
No known patch for locale af, 10.0 -> 14.0.1; skipping...
instead of generating snippets. This is because the checksums files have lines like this
26dee2baa2b425fb3d8057879558bf89 md5 7771770 update/win32/en-US/firefox-13.0.2-14.0.1.partial.mar
Which is the same problem with the wrong version in the config. In fact we have generated partials for 13.0.2 to 14.0.1, which won't be any use, and will need to create 13.0.1 to 14.0.1 partials.
 
==== Manually generate 13.0.1-14.0.1 partials ====
Working from the [https://wiki.mozilla.org/ReleaseEngineering/How_To/Generate_partial_updates howto doc], but no config changes are necessary here.
 
<pre>
<pre>
## on linux-ix-slave36
mkdir: cannot create directory `/pub/mozilla.org/firefox/nightly/14.0.1-candidates/build1//contrib': File exists
cd /builds/slave/rel-m-rel-updates/build
mkdir: cannot create directory `/pub/mozilla.org/firefox/nightly/14.0.1-candidates/build1//contrib-localized': File exists
mkdir -p logs/13.0.1-14.0.1
 
## build tools (9 min)
export HGROOT=http://hg.mozilla.org/releases/mozilla-release
perl patcher2.pl --build-tools-hg --tools-revision=FIREFOX_14_0_1_RELEASE \
  --app=firefox --brand=Firefox \
  --config=patcher-configs/mozRelease-branch-patcher2.cfg
 
## get complete mar files  (75 min, very badly needs parallelization)
perl patcher2.pl --download \
  --app=firefox --brand=Firefox \
  --config=patcher-configs/mozRelease-branch-patcher2.cfg \
  2>&1 | tee logs/13.0.1-14.0.1/download.log
 
## generate unsigned partial updates (100 min)
perl patcher2.pl --create-patches --partial-patchlist-file=patchlist.cfg \
--app=firefox --brand=Firefox \
--config=patcher-configs/mozRelease-branch-patcher2.cfg \
  2>&1 | tee logs/13.0.1-14.0.1/create-partials.log
 
## Make a backup of the new partials/snippets, destroy snippets (6 min)
rsync -a temp/firefox/13.0.1-14.0.1{,-unsigned}/
rm -rf temp/firefox/13.0.1-14.0.1/aus2*
 
## Sign the mar files (< 1 min)
# get a token
# look up the values for --user in the release-signing part of passwords.py, on the master
unset HISTFILE
IP=`/sbin/ifconfig eth0 | grep "inet addr" | awk -F: '{print $2}' | awk '{print $1}'`
ssh -oBatchMode=no cltbld@buildbot-master12.build.mozilla.org \
  "curl --fail -k -Fslave_ip=$IP -Fduration=3600 --user reda:cted \
  https://signing1.build.scl1.mozilla.com:9120/token" \
    > /builds/slave/rel-m-rel-updates/token
rm /builds/slave/rel-m-rel-updates/nonce
 
# update the channel id and version, then sign (25 min)
(for f in `find temp/firefox/13.0.1-14.0.1/ftp -type f -name *partial.mar | sort`; do
  echo Signing $f
  mozilla/obj/dist/host/bin/mar -H firefox-mozilla-release -V 14.0.1 -i $f;
  /tools/python-2.6.5/bin/python \
    /builds/slave/rel-m-rel-updates/tools/release/signing/signtool.py \
    -t /builds/slave/rel-m-rel-updates/token -n /builds/slave/rel-m-rel-updates/nonce \
    -c /builds/slave/rel-m-rel-updates/tools/release/signing/host.cert \
    -H signing1.build.scl1.mozilla.com:9120 -H signing2.build.scl1.mozilla.com:9120 -f gpg -f mar \
    "$f"
done ) 2>&1 | tee logs/13.0.1-14.0.1/sign-partials.log
 
## Regenerate the snippets (4 min)
perl patcher2.pl --create-patchinfo \
  --app=firefox --brand=Firefox \
  --config=patcher-configs/mozRelease-branch-patcher2.cfg \
  2>&1 | tee logs/13.0.1-14.0.1/generate_snippets.log
 
# Snippet verification
cd temp/firefox/13.0.1-14.0.1
bash /builds/slave/rel-m-rel-updates/tools/release/compare-channel-snippets.sh \
  aus2 release aus2.test  releasetest
 
# Tidy and upload mar files (6 min)
find ftp/firefox/nightly/14.0.1-candidates/build1 -type f -exec chmod 644 {} \;
find ftp/firefox/nightly/14.0.1-candidates/build1 -type d -exec chmod 755 {} \;
# $BUILD is the build number, eg 1, 2, ...
rsync -nav -e 'ssh -oIdentityFile=~/.ssh/ffxbld_dsa' '--exclude=*complete.mar' \
  ftp/firefox/nightly/14.0.1-candidates/build1/update \
  ffxbld@stage.mozilla.org:/pub/mozilla.org/firefox/nightly/14.0.1-candidates/build1/
# then repeat without the -n
 
# Upload snippets (2 min, NB: no  --exclude=complete.txt)
rsync -av -e 'ssh -oIdentityFile=~/.ssh/auspush' \
aus2.test/ \
ffxbld@aus3-staging.mozilla.org:/opt/aus2/snippets/staging/Firefox-14.0.1-build1-test
rsync -av -e 'ssh -oIdentityFile=~/.ssh/auspush' \
aus2/ \
ffxbld@aus3-staging.mozilla.org:/opt/aus2/snippets/staging/Firefox-14.0.1-build1
</pre>
</pre>


Defer backupsnip until after other changes.
That's OK though, because the SUMS files uploaded prior to that.


==== Adjust What's New in snippets ====
There are no checksums for the 13.0.1 partial MARs because they weren't generated on the build machines. I don't think we need to bother fixing that up (bhearsum).
From {{bug|767207}}
<pre>
locales="en-US en-GB fr it de es-ES es-AR es-CL es-MX ja ja-JP-mac ko ru"


# backup auto-generated snippets
I also noticed {{bug|774345}} while verifying the new files, which had given us a bunch of 13.0 checksums in the original run. That's not a problem now because they were overwritten with 14.0.1-only sums files.
cd /opt/aus2/snippets/staging
rsync -a Firefox-14.0.1-build1{,-withoutWN}/
rsync -a Firefox-14.0.1-build1-test{,-withoutWN}/


for l in $locales; do
=== Updates ===
  for f in `find Firefox-14.0.1-build1{,-test}/*/*/*/*/$l -type f`; do
See [[Releases/Firefox_14.0.1/BuildNotes-Updates]] because it's too long to keep here.
    echo tweaking $f
    sed -i '/^actions.*$/d' $f
  done
done
</pre>
Did some quick checks using diff, looks OK.
 
==== Generate 12.0 to 14.0.1 partial update for windows ====
[https://wiki.mozilla.org/ReleaseEngineering/How_To/Generate_partial_updates doc]
 
==== Generate release-cck-mozilla11/12 snippets ====
By duplicating the 13.0.1 release/releasetest snippets and modifying the bouncer product, see [https://bugzilla.mozilla.org/show_bug.cgi?id=758795#c8 bug] for more info.
 
==== Update verify ====


=== Reset reserved slaves ===
=== Reset reserved slaves ===
https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Tagging_through_Signing#Reset_reserved_slaves
# buildbot-master12
cd /builds/buildbot/build1/master/
echo 0 > reserved_slaves


=== Check permissions / AV scan ===
=== Check permissions / AV scan ===
No problems, but will need to rerun after 12.0 - 14.0.1 partial generation
No problems, but will need to rerun after 12.0 - 14.0.1 partial generation. Re-run had no problems either.


=== Push to internal mirrors ===
=== Push to internal mirrors ===
https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Push_to_releases_directory_.2F_internal_mirrors
https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Push_to_releases_directory_.2F_internal_mirrors


There is extra work to do here for funnelcake, see [https://bugzilla.mozilla.org/show_bug.cgi?id=758795#c8 the bug]
Move away the 13.0.2-14.0.1 partials that we aren't going to use:
# ffxbld@stage
mkdir -p ~/14.0.1-candidates-partials/build1/update
rsync -av --include=update --include=update/* --include=update/*/* \
  --include=update/*/*/firefox-13.0.2-14.0.1.partial.mar* --exclude=* \
  /pub/mozilla.org/firefox/candidates/14.0.1-candidates/build1/ ~/14.0.1-candidates-partials/build1/
  find /pub/mozilla.org/firefox/candidates/14.0.1-candidates/build1/ -type f \
    -name 'firefox-13.0.2-14.0.1.partial.mar*' -exec rm -v {} \;
 
Funnelcake11:
# ffxbld@stage
rsync -av --exclude=firefox-12.0-14.0.1.partial.mar* --exclude=index.html\
  /pub/mozilla.org/firefox/releases/14.0.1/update/win32 \
  /pub/mozilla.org/firefox/releases/13.0.1-funnelcake11/update/
rsync -av --exclude=firefox-12.0-14.0.1.partial.mar* --exclude=index.html\
  /pub/mozilla.org/firefox/releases/14.0.1/update/mac \
  /pub/mozilla.org/firefox/releases/13.0.1-funnelcake11/update/
 
Funnelcake12:
# ffxbld@stage
rsync -av --exclude=firefox-12.0-14.0.1.partial.mar* --exclude=index.html\
  /pub/mozilla.org/firefox/releases/14.0.1/update/win32 \
  /pub/mozilla.org/metrics/13.0.1-funnelcake12/update/
rsync -av --exclude=firefox-12.0-14.0.1.partial.mar* --exclude=index.html\
  /pub/mozilla.org/firefox/releases/14.0.1/update/mac \
  /pub/mozilla.org/metrics/13.0.1-funnelcake12/update/
 
Fix up EUballot:
# ffxbld@stage
rsync -av \
  /pub/mozilla.org/firefox/candidates/14.0.1-candidates/build1/partner-repacks/euballot/win32/ \
  /pub/mozilla.org/firefox/releases/14.0.1/win32-EUballot/


=== Final verification ===
=== Final verification ===
https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Final_Verification
Got extra runs from double-building the uptake monitor and confusing the master. A couple failed to clone build tools and were auto-retried. Reconfiging the master fixed repeated notifications of uptake on releasetest from triggering more jobs.


=== Before Release Day ===
=== Before Release Day ===


==== Run backupsnip for release snippets ====
==== Run backupsnip for release snippets ====
https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Run_Backupsnip
cd /opt/aus2/snippets/staging
~/bin/backupsnip Firefox-14.0.1-build1


=== Push index file to mirrors ===
=== Push index file to mirrors ===
https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Dealing_with_index.html_files
<pre>
# ffxbld@stage
version=14.0.1
cd /pub/mozilla.org/firefox/releases/$version
wget --no-check-certificate -O index.html https://bugzilla.mozilla.org/attachment.cgi?id=631778
version=14
sed -i -e "s/13/$version/g" index.html
for dir in `find . -mindepth 1 -type d `; do cp -pv index.html $dir/; done
</pre>


=== Push to external mirrors ===
=== Push to external mirrors ===
https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Push_to_external_mirrors
Unblocked firefox/releases/13.0.1-funnelcake/update as well firefox/releases/14.0.1/. Updated mozilla-current for FF 14.0.1 and TB 14.0.


Unblock firefox/releases/13.0.1-funnelcake/update as well firefox/releases/14.0.1/.
Had to force the uptake monitoring builder.


=== Release Day ===
=== Release Day ===
Line 428: Line 345:
https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Update_XULRunner_wiki_page
https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Update_XULRunner_wiki_page


==== Check Throttling  ====
Updated https://developer.mozilla.org/en/XULRunner and https://developer.mozilla.org/en/Gecko_SDK.
https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Check_Throttling


==== Push to Release Channel (for beta releases and release releases) ====
==== Push to Release Channel (for beta releases and release releases) ====
https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Push_snippets
https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Push_snippets


We'll have extra snippets to push for funnelcake, and the 12.0-14.0.1 partial. The correct order is
Pushed snippets (order was important, to make sure mac funnelcake partials were blocked):
* release-cck-mozilla11, release-cck-mozilla12
# ffxbld@aus3-staging
* main snippets
cd /opt/aus2/snippets/staging
* 12.0-14.0.1 partial
~/bin/pushsnip Firefox-14.0.1-build1-funnelcake11_12
~/bin/pushsnip Firefox-14.0.1-build1
 
Confirmed partials for 12.0 and 13.0.1:
* 12.0: https://aus3.mozilla.org/update/1/Firefox/12.0/20120420145725/WINNT_x86-msvc/en-US/release/update.xml?force=1
* 13.0.1: https://aus3.mozilla.org/update/1/Firefox/13.0.1/20120614114901/WINNT_x86-msvc/en-US/release/update.xml?force=1
 
Confirmed funnelcake mac partials:
* funnelcake11: https://aus3.mozilla.org/update/1/Firefox/13.0.1/20120614114901/Darwin_x86_64-gcc3-u-i386-x86_64/en-US/release-cck-mozilla11/update.xml
* funnelcake12: https://aus3.mozilla.org/update/1/Firefox/13.0.1/20120614114901/Darwin_x86_64-gcc3-u-i386-x86_64/en-US/release-cck-mozilla12/update.xml


==== Update symlinks ====
==== Update symlinks ====
https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Update_Symlinks
https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Update_Symlinks
As ffxbld@stage.mozilla.org:
cd /pub/mozilla.org/firefox/releases
rm latest && ln -s 14.0.1 latest


===== Remove index.html files =====
===== Remove index.html files =====
https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Dealing_with_index.html_files
https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Dealing_with_index.html_files
As ffxbld@stage.mozilla.org:
find /pub/mozilla.org/firefox/releases/14.0.1 -name index.html -exec rm {} \;


==== Update browser choice website ====
==== Update browser choice website ====
https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Update_the_Browser_Choice_Website
https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Update_the_Browser_Choice_Website
Filed {{bug|774687}}


==Build 2 (Fennec only)==
==Build 2 (Fennec only)==
Line 483: Line 417:
=====Push the files (for beta releases and release releases)=====
=====Push the files (for beta releases and release releases)=====
https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Push_the_files
https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Push_the_files
 
* Updated http://hg.mozilla.org/build/braindump/raw-file/tip/releases-related/push_fennec.sh to point at 14.0.1build2
* Ran the following on upload1:
curl -O http://hg.mozilla.org/build/braindump/raw-file/tip/releases-related/push_fennec.sh
bash push_fennec.sh
=====Play Store (for beta releases and release releases)=====
=====Play Store (for beta releases and release releases)=====
https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Play_Store
https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Play_Store
* Downloaded [http://ftp.mozilla.org/pub/mozilla.org/mobile/releases/14.0.1/android/multi/fennec-14.0.1.multi.android-arm.apk multi locale APK] to my laptop.
* Went to [https://market.android.com/publish android market]
* Clicked on "Firefox"
* Clicked "APK Files"
* Clicked "Upload APK", uploaded 14.0.1 multi APK.
* Clicked "Save"
* Clicked "Deactivate" on 14.0
* Clicked "Activate" on 14.0.1
* Clicked "Save"
Publishing happened at 7:10am PT. Confirmed that the caches had cleared at XXXX.

Latest revision as of 15:46, 17 July 2012

Notes About Releasing

Please update the Notes Template and the Release:Primer for future releases (bug fixes, changes to automation) as needed

Bugs hit

Enter any bugs pre-existing or newly discovered and filed during the release:

  • bug 773876 - partner repack's download_pkg-dmg step does not retry failed wget
  • bug 774601 - euballot builds put in wrong part of candidates directory

Build Engineers

nthomas with a start from bhearsum - Tracking bug: bug 758795

Signed-off Revision(s)

Build 1: 228ba1a111fc

L10N changesets

Instructions on how to get them

Tags

Manually tag the automation code, then record the generated tags below. (details)

Build # Branch, Tags Changeset
1 MOBILE1401_2012071313_RELBRANCH, FENNEC_14_0_1_RELEASE, FENNEC_14_0_1_BUILD1 9c64e364f5df
GECKO1401_2012071313_RELBRANCH, FIREFOX_14_0_1_BUILD1 FIREFOX_14_0_1_RELEASE e5728a4e106c
2 MOBILE1401_2012071316_RELBRANCH FENNEC_14_0_1_BUILD2 FENNEC_14_0_1_RELEASE 5ca6bd1f2bdf

Build data

Firefox

Build # Type Build ID Build machine Time to build
1 Linux 20120713134347 inux-ix-slave39 1 hrs, 9 mins, 10 secs
Linux64 linux64-ix-slave16 1 hrs, 4 mins, 38 secs
Mac bld-lion-r5-063 1 hrs, 19 mins, 34 secs
Windows mw32-ix-slave06 3 hrs, 41 mins, 16 secs

Fennec

Build # Type Build ID Build machine Time to build
1 android 20120713133725 linux-ix-slave06 50 mins, 41 secs
2 android 20120713170118 linux-ix-slave39 52 mins, 49 secs

Notes

Build 1

Diff mozconfigs

From a mozilla-release checkout (with 13.0.1 content), ran: 

for p in linux32 linux64 macosx-universal win32; do diff -Naur {,../mozilla-beta/}browser/config/mozconfigs/$p/release; done

(note: mozilla-beta had not been updated to 15.0 code yet.)

No differences, nothing to do here.

Set reserved slaves

  • set reserved_slaves to 8:
cd /builds/buildbot/build1
echo 8 > master/reserved_slaves

Preparing to start Automation

  • set a clobber for any master, release-mozilla-release, any builder as a speed up
  • create a symlink to the candidates directory
# ffxbld or tbirdbld@stage
version=14.0.1
product=firefox
cd /pub/mozilla.org/$product/nightly/
mkdir ../candidates/$version-candidates
ln -s ../candidates/$version-candidates $version-candidates
  • moved the slavealloc lock for 6 mw32-ix-slave from bm13 to bm12, rebooted them. mw32-ix-slave04 needed TLC for fuzzing exhausting e drive (bug 771687). Plenty of linux/linux64 slaves already. Locked 8 bld-r5-lion slaves to bm12.
  • Shipped the l10n milestones
    • Clicked "Ship" for Fennec
      • Clicked "Add"
        • repo: releases/mozilla-release
        • branch: default
        • path: mobile/android/locales/maemo-locales
      • Clicked "Ship it"
    • Clicked "Ship" for Firefox
      • Clicked "Ship it"
  • Landed buildbot-configs patch w/ updated l10n-changesets:
wget -O- --no-check-certificate "https://bugzilla.mozilla.org/attachment.cgi?id=641885" | patch -p1
sed -i -e "s/TBD/'228ba1a111fc'/" mozilla/release-firefox-mozilla-release.py mozilla/release-fennec-mozilla-release.py
wget -O mozilla/l10n-changesets_mobile-release.json "https://l10n-stage-sj.mozilla.org/shipping/json-changesets?ms=fennec14.0.1&platforms=android&multi_android-multilocale_repo=releases%2Fmozilla-beta&multi_android-multilocale_rev=default&multi_android-multilocale_path=mobile%2Fandroid%2Flocales%2Fmaemo-locales"
wget -O mozilla/l10n-changesets_mozilla-release "https://l10n-stage-sj.mozilla.org/shipping/l10n-changesets?ms=fx14.0.1"
# Remove the locales we aren't shipping from l10n changesets
sed -i -e '/^mn.*$/d' -e '/^sw.*$/d' mozilla/l10n-changesets_mozilla-release
hg commit -m "bug 758795: tracking bug for build and release of Firefox/Fennec 14.0.1. r=aki"
hg up -r production
hg transplant default
hg tag -r production {FIREFOX,FENNEC}_14_0_1_{RELEASE,BUILD1}
hg push ssh://hg.mozilla.org/build/buildbot-configs
  • Tagged buildbotcustom:
hg tag -r production-0.8 {FIREFOX,FENNEC}_14_0_1_{RELEASE,BUILD1}
hg push ssh://hg.mozilla.org/build/buildbotcustom
  • Tagged tools:
hg tag -r default {FIREFOX,FENNEC}_14_0_1_{RELEASE,BUILD1}
hg push ssh://hg.mozilla.org/build/tools
  • Updated and reconfiged buildbot-master12:
cd /builds/buildbot/build1
source bin/activate
make update reconfig
  • Ran release sanity in dry run mode:
cd /builds/buildbot/build1/master
source ../bin/activate
PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u bhearsum \
   -V 14.0.1 --branch mozilla-release --build-number 1 \
   --release-config release-firefox-mozilla-release.py \
   --release-config release-fennec-mozilla-release.py --products firefox,fennec  \
   --dryrun localhost:9001
    • Complained about mn and sw being missing, because the l10n dashboard is wrong
  • Ran release sanity with --bypass-l10n-check to get the release going:
PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u bhearsum \
   -V 14.0.1 --branch mozilla-release --build-number 1 \
   --release-config release-firefox-mozilla-release.py \
   --release-config release-fennec-mozilla-release.py --products firefox,fennec  \
   --bypass-l10n-check localhost:9001

E-mail Metrics

Sent the following mail to metrics-alerts@mozilla.org: 

Firefox,firefox,firefox,14.0.1,14.0
Fennec,firefox-mobile,fennec,14.0.1,14.0

E-mail mirrors@mozilla.org

Followed up to IT message about CDN usage.

Edit rsync exclude files

Committed this diff, after review: 

Index: rsyncd-mozilla-releases.exclude
===================================================================
--- rsyncd-mozilla-releases.exclude	(revision 43094)
+++ rsyncd-mozilla-releases.exclude	(working copy)
@@ -1,3 +1,12 @@
++ firefox
++ firefox/releases
++ firefox/releases/14.0.1
++ firefox/releases/14.0.1/*/
++ firefox/releases/14.0.1/*/*/
++ firefox/releases/14.0.1/*/*/*/
++ firefox/releases/14.0.1/**index.html
+- firefox/releases/14.0.1/**
+- firefox/releases/13.0.1-funnelcake11/update
 - firefox/releases/0.10
 - firefox/releases/0.10.1
 - firefox/releases/0.10rc

Tag

No problems

Bouncer Submitter

Bogus 13.0.2 oldVersion (see Updates for diagnosis) resulted in

  • renaming Firefox-14.0.1-Partial-13.0.2 to Firefox-14.0.1-Partial-13.0.1
  • s/13.0.2/13.0.1/ in the 8 locations for that product

Added Firefox-14.0.1-Partial-12.0 product and location for win, for that special partial.

Funnelcake11:

  • added products Firefox-14.0.1-Partial-13.0.1-Funnelcake11 & Firefox-14.0.1-Complete-Funnelcake11
  • added win and osx locations for both

Funnelcake12:

  • added products Firefox-14.0.1-Partial-13.0.1-Funnelcake12 & Firefox-14.0.1-Complete-Funnelcake12
  • added win and osx locations for both

(both of these should be one-off for this release)

Source

Failed because I removed the buildbot-configs mozconfig that this step depends on. Reverted with: 

hg revert -r 6507 mozilla2/linux/mozilla-release/release/mozconfig
hg commit -m "Re-add linux mozconfig so source steps will work"
hg up -r production
hg transplant -r production
hg tag -f -r production {FIREFOX,FENNEC}_14_0_1_{RELEASE,BUILD1}
hg push ssh://hg.mozilla.org/build/buildbot-configs

Used "rebuild" to restart source steps.

Failed again with a configure error. Realized this was because I didn't merge the beta mozconfig -> production. Brought forward compiler changes in http://hg.mozilla.org/build/buildbot-configs/rev/466340b368b8 to fix this. Used rebuild to try again.

Failed again with the same error. I did some debugging on the machine and found that there were some CFLAGS/CXXFLAGS I didn't bring forward and did that in http://hg.mozilla.org/build/buildbot-configs/rev/b2b6a37879f2. Used rebuild to try again. Worked this time.

Build

Firefox

No problems

Firefox repacks

No problems

Fennec

No problems

Fennec repacks

No problems

XULRunner

Source

Failed with the same issue as Firefox source builder.

Build

  • Mac build failed first time because of bug 753223. Rebuilt it, got success
  • Other platforms no problems

Push to Mirrors

No problems

Partner Repack

Firefox

Mac jobs failed when it got a 500 error trying to d/l pkg-dmg from hg.m.o, bug 773876 filed. Success on rebuild. EUballot builds went to wrong place - bug 774601. Otherwise fine.

Funnelcake builds are documented elsewhere.

Android Signing

FENNEC ONLY RESPIN DECLARED.

Checksums

No problems, but will need to rerun for 12.0 to 14.0.1 partials.

On the 2nd run, failed with: 

mkdir: cannot create directory `/pub/mozilla.org/firefox/nightly/14.0.1-candidates/build1//contrib': File exists
mkdir: cannot create directory `/pub/mozilla.org/firefox/nightly/14.0.1-candidates/build1//contrib-localized': File exists

That's OK though, because the SUMS files uploaded prior to that.

There are no checksums for the 13.0.1 partial MARs because they weren't generated on the build machines. I don't think we need to bother fixing that up (bhearsum).

I also noticed bug 774345 while verifying the new files, which had given us a bunch of 13.0 checksums in the original run. That's not a problem now because they were overwritten with 14.0.1-only sums files.

Updates

See Releases/Firefox_14.0.1/BuildNotes-Updates because it's too long to keep here.

Reset reserved slaves

# buildbot-master12
cd /builds/buildbot/build1/master/
echo 0 > reserved_slaves

Check permissions / AV scan

No problems, but will need to rerun after 12.0 - 14.0.1 partial generation. Re-run had no problems either.

Push to internal mirrors

https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Push_to_releases_directory_.2F_internal_mirrors

Move away the 13.0.2-14.0.1 partials that we aren't going to use: 

# ffxbld@stage
mkdir -p ~/14.0.1-candidates-partials/build1/update
rsync -av --include=update --include=update/* --include=update/*/* \
  --include=update/*/*/firefox-13.0.2-14.0.1.partial.mar* --exclude=* \
 /pub/mozilla.org/firefox/candidates/14.0.1-candidates/build1/ ~/14.0.1-candidates-partials/build1/ 
 find /pub/mozilla.org/firefox/candidates/14.0.1-candidates/build1/ -type f \
   -name 'firefox-13.0.2-14.0.1.partial.mar*' -exec rm -v {} \;

Funnelcake11: 

# ffxbld@stage
rsync -av --exclude=firefox-12.0-14.0.1.partial.mar* --exclude=index.html\
  /pub/mozilla.org/firefox/releases/14.0.1/update/win32 \
  /pub/mozilla.org/firefox/releases/13.0.1-funnelcake11/update/
rsync -av --exclude=firefox-12.0-14.0.1.partial.mar* --exclude=index.html\
  /pub/mozilla.org/firefox/releases/14.0.1/update/mac \
  /pub/mozilla.org/firefox/releases/13.0.1-funnelcake11/update/

Funnelcake12: 

# ffxbld@stage
rsync -av --exclude=firefox-12.0-14.0.1.partial.mar* --exclude=index.html\
  /pub/mozilla.org/firefox/releases/14.0.1/update/win32 \
  /pub/mozilla.org/metrics/13.0.1-funnelcake12/update/
rsync -av --exclude=firefox-12.0-14.0.1.partial.mar* --exclude=index.html\
  /pub/mozilla.org/firefox/releases/14.0.1/update/mac \
  /pub/mozilla.org/metrics/13.0.1-funnelcake12/update/

Fix up EUballot: 

# ffxbld@stage
rsync -av \
  /pub/mozilla.org/firefox/candidates/14.0.1-candidates/build1/partner-repacks/euballot/win32/ \
  /pub/mozilla.org/firefox/releases/14.0.1/win32-EUballot/

Final verification

Got extra runs from double-building the uptake monitor and confusing the master. A couple failed to clone build tools and were auto-retried. Reconfiging the master fixed repeated notifications of uptake on releasetest from triggering more jobs.

Before Release Day

Run backupsnip for release snippets

cd /opt/aus2/snippets/staging
~/bin/backupsnip Firefox-14.0.1-build1

Push index file to mirrors

# ffxbld@stage
version=14.0.1
cd /pub/mozilla.org/firefox/releases/$version
wget --no-check-certificate -O index.html https://bugzilla.mozilla.org/attachment.cgi?id=631778
version=14
sed -i -e "s/13/$version/g" index.html 
for dir in `find . -mindepth 1 -type d `; do cp -pv index.html $dir/; done

Push to external mirrors

Unblocked firefox/releases/13.0.1-funnelcake/update as well firefox/releases/14.0.1/. Updated mozilla-current for FF 14.0.1 and TB 14.0.

Had to force the uptake monitoring builder.

Release Day

Update XULRunner wiki page

https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Update_XULRunner_wiki_page

Updated https://developer.mozilla.org/en/XULRunner and https://developer.mozilla.org/en/Gecko_SDK.

Push to Release Channel (for beta releases and release releases)

https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Push_snippets

Pushed snippets (order was important, to make sure mac funnelcake partials were blocked): 

# ffxbld@aus3-staging
cd /opt/aus2/snippets/staging
~/bin/pushsnip Firefox-14.0.1-build1-funnelcake11_12
~/bin/pushsnip Firefox-14.0.1-build1

Confirmed partials for 12.0 and 13.0.1:

Confirmed funnelcake mac partials:

Update symlinks

https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Update_Symlinks

As ffxbld@stage.mozilla.org: 

cd /pub/mozilla.org/firefox/releases
rm latest && ln -s 14.0.1 latest
Remove index.html files

https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Dealing_with_index.html_files

As ffxbld@stage.mozilla.org: 

find /pub/mozilla.org/firefox/releases/14.0.1 -name index.html -exec rm {} \;

Update browser choice website

https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Update_the_Browser_Choice_Website

Filed bug 774687

Build 2 (Fennec only)

Preparing to start Automation

  • Landed config update
  • Retagged with FENNEC_14_0_1_{BUILD2,RELEASE}
  • Updated, reconfiged.
  • Ran release sanity, passed:
cd /builds/buildbot/build1
source bin/activae
PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u bhearsum     -V 14.0.1 --branch mozilla-release --build-number 2 --release-config release-fennec-mozilla-release.py --products fennec      --dryrun localhost:9001
  • Ran without dryrun to start:
PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u bhearsum     -V 14.0.1 --branch mozilla-release --build-number 2 --release-config release-fennec-mozilla-release.py --products fennec  localhost:9001

Tag

No problems

Source

No problems

Build

Fennec

No problems

Fennec repacks

No problems

Android Signing

# cltsign@signing1
cd ~/signing-work/mozharness
hg pull -u && hg up -C -r default
python2.6 ~/signing-work/mozharness/scripts/sign_android.py --config-file signing/android_mozilla-release.py --platform android

Release Day

Publish Fennec to the Play Store (for beta releases and release releases)

Push the files (for beta releases and release releases)

https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Push_the_files 

curl -O http://hg.mozilla.org/build/braindump/raw-file/tip/releases-related/push_fennec.sh
bash push_fennec.sh
Play Store (for beta releases and release releases)

https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Play_Store

  • Downloaded multi locale APK to my laptop.
  • Went to android market
  • Clicked on "Firefox"
  • Clicked "APK Files"
  • Clicked "Upload APK", uploaded 14.0.1 multi APK.
  • Clicked "Save"
  • Clicked "Deactivate" on 14.0
  • Clicked "Activate" on 14.0.1
  • Clicked "Save"

Publishing happened at 7:10am PT. Confirmed that the caches had cleared at XXXX.

Retrieved from "https://wiki.allizom.org/index.php?title=Releases/Firefox_14.0.1/BuildNotes&oldid=451546"