Releases/Firefox 2.0.0.20 MU 2/BuildNotes

From MozillaWiki
Jump to: navigation, search

Release Engineers

Nick Thomas

Bugs

bug 487521 - tracking bug
bug ??? - nulls/refreshes for partners (not needed)

2.0.0.20 to 3.0.9

Generate Major Update Snippets

These are for the "vanilla" builds we're going to update.

Update these config files

  • CVS:mozilla/tools/patcher-configs/moz18-branch-major-update-patcher2.cfg
  • HG: build/tools/release/updates/moz18-firefox-{linux,mac,win32}-major.cfg

with review as usual.

Then generate the snippets

# ssh cltbld@production-prometheus-vm.build.mozilla.org
mkdir -p /builds/2.0.0.20-3.0.9-major-update/snippets
cd /builds/2.0.0.20-3.0.9-major-update/snippets
cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R7 -d patcher mozilla/tools/patcher
cd patcher
cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R7 -d MozBuild mozilla/tools/release/MozBuild
cd ../
cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -d patcher-configs mozilla/tools/patcher-configs

cd patcher

export CVSROOT=:ext:cltbld@cvs.mozilla.org:/cvsroot
  1. build tools
./patcher2.pl --build-tools --app=firefox --tools-rev=UPDATE_PACKAGING_R7 --config=../patcher-configs/moz18-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_build-tools.log
  1. download mars
./patcher2.pl --download --app=firefox --config=../patcher-configs/moz18-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_download.log
  1. FIXME - patcher needs to see that the MARs that it thinks are partials
#             are there or else it will not attempt to generate patchinfo
cd temp/firefox
ln -s 3.0.9 2.0.0.20-3.0.9
cd ../..
  1. Create partial patches and snippets
./patcher2.pl --create-patches --app=firefox --config=../patcher-configs/moz18-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox-create-patches.log

Separate out beta snippets

cd temp/firefox/2.0.0.20-3.0.9
rsync -a aus2/ aus2.beta/
find aus2.beta/ -type d -name "release" -exec rm -rfv {} \;
find aus2/ -type d -name "beta" -exec rm -rfv {} \;

Quick verify

This has changed a bit from last time because we haven't shipped 3.0.9 yet. Therefore we have to do QA using betatest pointing at the candidates dir, then 3.0.9 ships and we later go to beta (using bouncer), and later still enable the release channel (bouncer again). So we want releasetest == beta == release and accept that betatest is different.

  1. releasetest == beta
find aus2.test -type d -iregex '.releasetest.' | perl -nle '$a = $_; $a =~ s/releasetest/beta/; $a =~ s/aus2.test/aus2.beta/; system("diff -r -u $_ $a");'
find aus2.beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/releasetest/; $a =~ s/aus2.beta/aus2.test/; system("diff -r -u $_ $a");'
  1. releasetest == release
find aus2.test -type d -iregex '.releasetest.' | perl -nle '$a = $_; $a =~ s/releasetest/release/; $a =~ s/aus2.test/aus2/; system("diff -r -u $_ $a");'
find aus2 -type d -iregex '.*/release$' | perl -nle '$a = $_; $a =~ s/release$/releasetest/; $a =~ s/aus2/aus2.test/; system("diff -r -u $_ $a");'

No differences found.

Push to AUS

cd /builds/2.0.0.20-3.0.9-major-update/snippets/patcher/temp/firefox/2.0.0.20-3.0.9
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20090414-Firefox-2.0.0.20-3.0.9-MU
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.beta/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20090414-Firefox-2.0.0.20-3.0.9-MU-beta
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20090414-Firefox-2.0.0.20-3.0.9-MU-test

Enable test snippets

Safe to do this without doing null snippets because we're only talking about betatest and releasetest, where no end-users of partner builds will be falling back to (metrics website confirms).

~/bin/backupsnip 20090414-Firefox-2.0.0.20-3.0.9-MU-test
~/bin/pushsnip 20090414-Firefox-2.0.0.20-3.0.9-MU-test

Update verify

Ran update verify on fx-{linux,mac,win32}-1.9-slave2 as follows:

mkdir -p /builds/verify/firefox-20020-309-major/
cd /builds/verify/firefox-20020-309-major/
hg clone http://hg.mozilla.org/build/tools/
  1. get patch to trawl "Only in" directories
cd tools/release/common
# on mac & linux
curl -sL https://bugzilla.mozilla.org/attachment.cgi?id=367544 | patch -p3
# on win32
wget --no-check-certificate -L -O patch https://bugzilla.mozilla.org/attachment.cgi?id=367544
patch -p3 < patch
  1. everyone ...
platform=linux   # or mac or win32
cd ../updates
./verify.sh -c moz18-firefox-$platform-major.cfg 2>&1 | tee $platform.log

Logs look fine, and unchanged from last time. Each platform has some standard differences, here's Linux

diff -r source/firefox/defaults/pref/channel-prefs.js target/firefox/defaults/pref/channel-prefs.js
1c1
< //@line 2 "/builds/tinderbox/Fx-Mozilla1.8-release/Linux_2.4.21-53.EL_Depend/mozilla/browser/app/profile/channel-prefs.js"

//@line 2 "/builds/tinderbox/Fx-Mozilla1.9-Release/Linux_2.6.18-53.1.13.el5_Depend/mozilla/browser/app/profile/channel-prefs.js" Only in source/firefox/extensions: inspector@mozilla.org Only in source/firefox/extensions: talkback@mozilla.org Contents of source/firefox/extensions/inspector@mozilla.org dir only in source or target 442669 4 drwxr-xr-x 2 cltbld cltbld 4096 Apr 15 00:22 source/firefox/extensions/inspector@mozilla.org/components 442671 4 drwxr-xr-x 3 cltbld cltbld 4096 Dec 17 10:13 source/firefox/extensions/inspector@mozilla.org/defaults 442672 4 drwxr-xr-x 2 cltbld cltbld 4096 Apr 15 00:22 source/firefox/extensions/inspector@mozilla.org/defaults/preferences 442670 4 drwxr-xr-x 2 cltbld cltbld 4096 Apr 15 00:22 source/firefox/extensions/inspector@mozilla.org/chrome Contents of source/firefox/extensions/talkback@mozilla.org dir only in source or target 442674 4 drwxr-xr-x 3 cltbld cltbld 4096 Apr 15 00:22 source/firefox/extensions/talkback@mozilla.org/components 442675 4 drwxr-xr-x 2 cltbld cltbld 4096 Apr 15 00:22 source/firefox/extensions/talkback@mozilla.org/components/talkback WARN: non-binary files found in diff WARN: check_updates returned warning for Linux_x86-gcc3 downloads/firefox-2.0.0.20.tar.gz vs. downloads/firefox-3.0.9.af.linux-i686.tar.bz2: 2

  • The channel-prefs.js is a harmless comment change due the path on the build machine changing and us exlicitly not including that file in the complete mar
  • extensions/{inspector@mozilla.org,talkback@mozilla.org} are trees of empty dirs. We've removed all the files but the updater can't remove dirs (for safety), no problem because the Addons manager does that for us
  • Some locales also give messages like (eg be), due to them removing those plugins from the 3.0 build but we can't remove them for only some locales. Better user experience if the stuff you use is not vanished anyway Only in source/firefox/searchplugins: amazondotcom.xml Only in source/firefox/searchplugins: answers.xml Only in source/firefox/searchplugins: creativecommons.xml Only in source/firefox/searchplugins: eBay.xml

Mac

diff -r source/Firefox.app/Contents/MacOS/defaults/pref/channel-prefs.js target/Firefox.app/Contents/MacOS/defaults/pref/channel-prefs.js
1c1
< //@line 2 "/builds/tinderbox/Fx-Mozilla1.8-release/Darwin_8.7.0_Depend/mozilla/browser/app/profile/channel-prefs.js"

//@line 2 "/builds/tinderbox/Fx-Mozilla1.9-Release/Darwin_8.8.4_Depend/mozilla/browser/app/profile/channel-prefs.js" Only in source/Firefox.app/Contents/MacOS/extensions: inspector@mozilla.org Only in source/Firefox.app/Contents/MacOS/extensions: talkback@mozilla.org Only in source/Firefox.app/Contents/MacOS: init.d Only in source/Firefox.app/Contents/MacOS: updates Contents of source/Firefox.app/Contents/MacOS/extensions/inspector@mozilla.org dir only in source or target 1482817926 0 drwxr-xr-x 2 cltbld cltbld 68 Apr 15 00:25 source/Firefox.app/Contents/MacOS/extensions/inspector@mozilla.org/chrome 1482817927 0 drwxr-xr-x 2 cltbld cltbld 68 Apr 15 00:25 source/Firefox.app/Contents/MacOS/extensions/inspector@mozilla.org/components 1482817928 0 drwxr-xr-x 3 cltbld cltbld 102 Dec 17 11:32 source/Firefox.app/Contents/MacOS/extensions/inspector@mozilla.org/defaults 1482817930 0 drwxr-xr-x 2 cltbld cltbld 68 Apr 15 00:25 source/Firefox.app/Contents/MacOS/extensions/inspector@mozilla.org/defaults/preferences Contents of source/Firefox.app/Contents/MacOS/extensions/talkback@mozilla.org dir only in source or target 1482817932 0 drwxr-xr-x 3 cltbld cltbld 102 Apr 15 00:25 source/Firefox.app/Contents/MacOS/extensions/talkback@mozilla.org/components 1482817933 0 drwxr-xr-x 3 cltbld cltbld 102 Apr 15 00:25 source/Firefox.app/Contents/MacOS/extensions/talkback@mozilla.org/components/talkback 1482817934 0 drwxr-xr-x 3 cltbld cltbld 102 Dec 17 11:32 source/Firefox.app/Contents/MacOS/extensions/talkback@mozilla.org/components/talkback/Talkback.app 1482817935 0 drwxr-xr-x 4 cltbld cltbld 136 Apr 15 00:25 source/Firefox.app/Contents/MacOS/extensions/talkback@mozilla.org/components/talkback/Talkback.app/Contents 1482817936 0 drwxr-xr-x 2 cltbld cltbld 68 Apr 15 00:25 source/Firefox.app/Contents/MacOS/extensions/talkback@mozilla.org/components/talkback/Talkback.app/Contents/MacOS 1482817937 0 drwxr-xr-x 3 cltbld cltbld 102 Apr 15 00:25 source/Firefox.app/Contents/MacOS/extensions/talkback@mozilla.org/components/talkback/Talkback.app/Contents/Resources 1482817938 0 drwxr-xr-x 7 cltbld cltbld 238 Apr 15 00:25 source/Firefox.app/Contents/MacOS/extensions/talkback@mozilla.org/components/talkback/Talkback.app/Contents/Resources/English.lproj 1482817939 0 drwxr-xr-x 2 cltbld cltbld 68 Apr 15 00:25 source/Firefox.app/Contents/MacOS/extensions/talkback@mozilla.org/components/talkback/Talkback.app/Contents/Resources/English.lproj/ArchivingSettings.nib 1482817940 0 drwxr-xr-x 2 cltbld cltbld 68 Apr 15 00:25 source/Firefox.app/Contents/MacOS/extensions/talkback@mozilla.org/components/talkback/Talkback.app/Contents/Resources/English.lproj/IntroWizard.nib 1482817941 0 drwxr-xr-x 2 cltbld cltbld 68 Apr 15 00:25 source/Firefox.app/Contents/MacOS/extensions/talkback@mozilla.org/components/talkback/Talkback.app/Contents/Resources/English.lproj/MainMenu.nib 1482817942 0 drwxr-xr-x 2 cltbld cltbld 68 Apr 15 00:25 source/Firefox.app/Contents/MacOS/extensions/talkback@mozilla.org/components/talkback/Talkback.app/Contents/Resources/English.lproj/ProxySettings.nib 1482817943 0 drwxr-xr-x 2 cltbld cltbld 68 Apr 15 00:25 source/Firefox.app/Contents/MacOS/extensions/talkback@mozilla.org/components/talkback/Talkback.app/Contents/Resources/English.lproj/SendingSettings.nib Contents of source/Firefox.app/Contents/MacOS/init.d dir only in source or target Contents of source/Firefox.app/Contents/MacOS/updates dir only in source or target 1482817979 0 drwxr-xr-x 2 cltbld cltbld 68 Dec 17 11:32 source/Firefox.app/Contents/MacOS/updates/0 WARN: non-binary files found in diff WARN: check_updates returned warning for Darwin_Universal-gcc3 downloads/Firefox 2.0.0.20.dmg vs. downloads/firefox-3.0.9.af.mac.dmg: 2 * the same again, there are more dirs for talkback (.nib) * empty init.d dir can't be removed but is harmless * updates dir is from testing method I think

Win32

Only in source/bin: .autoreg
diff -r source/bin/defaults/pref/channel-prefs.js target/bin/defaults/pref/channel-prefs.js
1c1
< //@line 2 "/cygdrive/c/builds/tinderbox/Fx-Mozilla1.8-Release/WINNT_5.2_Depend/mozilla/browser/app/profile/channel-prefs.js"

//@line 2 "e:\fx19rel\WINNT_5.2_Depend\mozilla\browser\app\profile\channel-prefs.js" Only in source/bin/extensions: inspector@mozilla.org Only in source/bin/extensions: talkback@mozilla.org Binary files source/bin/freebl3.chk and target/bin/freebl3.chk differ Only in source/bin: removed-files Binary files source/bin/softokn3.chk and target/bin/softokn3.chk differ Contents of source/bin/extensions/inspector@mozilla.org dir only in source or target 239195 0 drwxr-xr-x 3 cltbld Administ 0 Apr 15 00:27 source/bin/extensions/inspector@mozilla.org/chrome 239196 0 drwxr-xr-x 3 cltbld Administ 0 Apr 15 00:27 source/bin/extensions/inspector@mozilla.org/chrome/icons 239197 0 drwxr-xr-x 2 cltbld Administ 0 Apr 15 00:27 source/bin/extensions/inspector@mozilla.org/chrome/icons/default 239201 0 drwxr-xr-x 2 cltbld Administ 0 Apr 15 00:27 source/bin/extensions/inspector@mozilla.org/components 239203 0 drwxr-xr-x 3 cltbld Administ 0 Apr 15 00:27 source/bin/extensions/inspector@mozilla.org/defaults 239204 0 drwxr-xr-x 2 cltbld Administ 0 Apr 15 00:27 source/bin/extensions/inspector@mozilla.org/defaults/preferences Contents of source/bin/extensions/talkback@mozilla.org dir only in source or target 239209 0 drwxr-xr-x 2 cltbld Administ 0 Apr 15 00:27 source/bin/extensions/talkback@mozilla.org/components FAIL: binary files found in diff FAIL: check_updates returned failure for WINNT_x86-msvc downloads/Firefox Setup 2.0.0.20.exe vs. downloads/firefox-3.0.9.af.win32.installer.exe: 1 * .autoreg is normal difference when updating (required?) * freebl3.chk and softoke3.chk diff is ok, still valid check files bug 404340 * removed-files is for the regeneration of uninstall logs (IIRC)

Announced to r-d

Update verify on releasetest

As we're using files in different locations, verify releasetest (same as beta) snippets are OK on fx-{linux,mac,win32}-1.9-slave2:

cd /builds/verify/firefox-20020-309-major/tools/release/updates
platform=linux   # or mac or win32
# modify moz18-firefox-$platform-major.cfg:
#  s/betatest/releasetest/
#  change to= to point to release builds on stage
./verify.sh -c moz18-firefox-$platform-major.cfg 2>&1 | tee $platform-releasetest.log

Linux passed OK, get timestamp and d/l location differences between betatest and releasetest logs. Had a bunch of trouble with slow mirrors (in particular mozilla.isohunt.com) for win32 and mac, so killed some wget processes (since we test twice per locale). This screws up diffing betatest and releasetest logs. Ran quick verify (./verify.sh -t instead of -c) for each platform to verify bouncer set up OK.

Enable beta snippets

Safe to do this without doing null snippets because there are no end-users of partner builds on beta channel (kev and metrics website confirmed last MU).

cd /opt/aus2/snippets/staging

time ~/bin/backupsnip 20090414-Firefox-2.0.0.20-3.0.9-MU-beta
real    39m15.394s
user    0m37.533s
sys     0m46.934s

time ~/bin/pushsnip 20090414-Firefox-2.0.0.20-3.0.9-MU-beta

real    0m4.426s
user    0m0.013s
sys     0m0.369s

At this point we had to firedrill 3.0.10 for a top crasher

2.0.0.20 to 3.0.10

Generate Major Update Snippets

These are for the "vanilla" builds we're going to update.

Update these config files

  • CVS:mozilla/tools/patcher-configs/moz18-branch-major-update-patcher2.cfg
  • HG: build/tools/release/updates/moz18-firefox-{linux,mac,win32}-major.cfg

with review as usual.

Then generate the snippets

# ssh cltbld@production-prometheus-vm.build.mozilla.org
mkdir -p /builds/2.0.0.20-3.0.10-major-update/snippets
cd /builds/2.0.0.20-3.0.10-major-update/snippets
cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R7 -d patcher mozilla/tools/patcher
cd patcher
cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R7 -d MozBuild mozilla/tools/release/MozBuild
cd ../
cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -d patcher-configs mozilla/tools/patcher-configs

cd patcher

export CVSROOT=:ext:cltbld@cvs.mozilla.org:/cvsroot
  1. build tools
./patcher2.pl --build-tools --app=firefox --tools-rev=UPDATE_PACKAGING_R7 --config=../patcher-configs/moz18-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_build-tools.log
  1. download mars
./patcher2.pl --download --app=firefox --config=../patcher-configs/moz18-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_download.log
  1. FIXME - patcher needs to see that the MARs that it thinks are partials
#             are there or else it will not attempt to generate patchinfo
cd temp/firefox
ln -s 3.0.10 2.0.0.20-3.0.10
cd ../..
  1. Create partial patches and snippets
./patcher2.pl --create-patches --app=firefox --config=../patcher-configs/moz18-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox-create-patches.log

Separate out beta snippets

cd temp/firefox/2.0.0.20-3.0.10
rsync -a aus2/ aus2.beta/
find aus2.beta/ -type d -name "release" -exec rm -rfv {} \;
find aus2/ -type d -name "beta" -exec rm -rfv {} \;

Quick verify

Back to what we did for the last major update, as 3.0.10 already shipped. Check the snippets are identical for all four channels

# betatest == releasetest
find aus2.test -type d -iregex '.betatest.' | perl -nle '$a = $_; $a =~ s/betatest/releasetest/; system("diff -r -u $_ $a");'
# test the opposite direction to catch anything that only exists for releasetest
find aus2.test -type d -iregex '.releasetest.' | perl -nle '$a = $_; $a =~ s/releasetest/betatest/; system("diff -r -u $_ $a");'
  1. betatest == beta
find aus2.test -type d -iregex '.betatest.' | perl -nle '$a = $_; $a =~ s/betatest/beta/; $a =~ s/aus2.test/aus2.beta/; system("diff -r -u $_ $a");'
find aus2.beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/betatest/; $a =~ s/aus2.beta/aus2.test/; system("diff -r -u $_ $a");'
  1. betatest == release
find aus2.test -type d -iregex '.betatest.' | perl -nle '$a = $_; $a =~ s/betatest/release/; $a =~ s/aus2.test/aus2/; system("diff -r -u $_ $a");'
find aus2 -type d -iregex '.*/release$' | perl -nle '$a = $_; $a =~ s/release$/betatest/; $a =~ s/aus2/aus2.test/; system("diff -r -u $_ $a");'

No differences detected.

Push to AUS

cd /builds/2.0.0.20-3.0.10-major-update/snippets/patcher/temp/firefox/2.0.0.20-3.0.10
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20090506-Firefox-2.0.0.20-3.0.10-MU
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.beta/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20090506-Firefox-2.0.0.20-3.0.10-MU-beta
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20090506-Firefox-2.0.0.20-3.0.10-MU-test

Enable test snippets

Safe to do this without doing null snippets because we're only talking about betatest and releasetest, where no end-users of partner builds will be falling back to (metrics website confirms).

~/bin/backupsnip 20090506-Firefox-2.0.0.20-3.0.10-MU-test
~/bin/pushsnip 20090506-Firefox-2.0.0.20-3.0.10-MU-test

Update verify

Ran update verify on fx-{linux,mac,win32}-1.9-slave2 as follows:

mkdir -p /builds/verify/firefox-20020-3010-major/
cd /builds/verify/firefox-20020-3010-major/
hg clone http://hg.mozilla.org/build/tools/
  1. get patch to trawl "Only in" directories
cd tools/release/common
# on mac & linux
curl -sL https://bugzilla.mozilla.org/attachment.cgi?id=367544 | patch -p3
# on win32
wget --no-check-certificate -L -O patch https://bugzilla.mozilla.org/attachment.cgi?id=367544
patch -p3 < patch
  1. everyone ...
platform=linux   # or mac or win32
cd ../updates
./verify.sh -c moz18-firefox-$platform-major.cfg 2>&1 | tee $platform.log

No regressions since 3.0.9 check (by eyeballing opendiff of logs). Announced to r-d.

Enable beta snippets

Safe to do this without doing null snippets because there are no end-users of partner builds on beta channel (kev and metrics website confirmed last MU).

time ~/bin/backupsnip 20090506-Firefox-2.0.0.20-3.0.10-MU-beta

real    41m53.805s
user    0m38.433s
sys     0m54.688s

time ~/bin/pushsnip 20090506-Firefox-2.0.0.20-3.0.10-MU-beta

real    0m4.893s
user    0m0.008s
sys     0m0.386s

Generate Null Partner Snippets

Need to block those partners which currently don't have a MU configured.

Create snippets

Nothing to do here - we started off with a full set of nulls, then either overwrote with valid major update for partners, or revoked a partner major update by putting back in nulls.

Push live

Nothing to do

Verify nulls

We should still check though, using the list of URLs we generated last time.

  1. cltbld@prometheus-vm
cd /builds/2.0.0.20-major-update/snippets/patcher/one-offs
./test_null_updates.sh urls-win32-release.txt 2>&1 | tee test-nulls-win32-release.txt
grep -v '^PASS' test-nulls-win32-release.txt
** FAIL  https://aus2.mozilla.org/update/1/Firefox/2.0.0.20/2008121709/WINNT_x86-msvc/en-US/release-cck-yahoo/update.xml
** FAIL  https://aus2.mozilla.org/update/1/Firefox/2.0.0.20/2008121709/WINNT_x86-msvc/ko/release-cck-yahoo/update.xml
** FAIL  https://aus2.mozilla.org/update/1/Firefox/2.0.0.20/2008121709/WINNT_x86-msvc/zh-CN/release-cck-yahoo/update.xml
** FAIL  https://aus2.mozilla.org/update/1/Firefox/2.0.0.20/2008121709/WINNT_x86-msvc/zh-TW/release-cck-yahoo/update.xml

Which is expected, those MU are live and therefore non-null info returned.

Repeat for each other two platforms, get no failures.

Enable release snippets

For the vanilla builds.

date; time ~/bin/backupsnip 20090506-Firefox-2.0.0.20-3.0.10-MU

real    48m35.745s
user    0m41.293s
sys     1m10.952s

time ~/bin/pushsnip 20090506-Firefox-2.0.0.20-3.0.10-MU

real    0m8.127s
user    0m0.008s
sys     0m0.427s