MozillaWiki

Releases/Firefox 4.0/BuildNotes

< Releases‎ | Firefox 4.0
Revision as of 03:08, 22 March 2011 by NThomas (talk | contribs) (→‎Major Updates)

Release Engineers

bhearsum, nthomas, jhford

Tracking Bug

bug 642234

Notes

Notify Mirrors

Joduinn sent this on March 16th.

RC1 version

Config Update

This small update to the release config was landed.

Automated Parts

Some of this release-day work makes sense to do from the release automation. Use force build for these.

Tag

forced build using the webform with values 

your name: jhford
reason for build: retagging for 4.0 release
branch to build: <left blank>
revision to build: <left blank>
repository to build: <left blank>
project to build: <left blank>
prop1 name: script_repo_revision
     value: FIREFOX_4_0_RELEASE
prop2 name: release_config
     value: mozilla/release-firefox-mozilla-2.0.py

Source

Forced build on builder page using form values 

your name: jhford
reason for build: retagging for 4.0 release
branch to build: <left blank>
revision to build: <left blank>
repository to build: <left blank>
project to build: <left blank>
prop1 name: <left blank>
     value: <left blank>
prop2 name: <left blank>
     value: <left blank>

Stage and Rename Files

Summary:

  • Copy files to ~ffxbld and rename to non-RC versions
    • Exclude *checksums* because we don't care for individual locale+platform checksums files in the final release dir
    • Exclude partial MARs because there's no use for them, no point in pushing them to the mirrors
    • Exclude Source packages because they're regenerated for final
    • Exclude *SUMS* because we'll be regenerating them.

Details: 

# ffxbld@stage
mkdir firefox-4.0
cd firefox-4.0
rsync -av --exclude=*checksums* --exclude='*partial.mar' --exclude=source --exclude=*SUMS* /pub/mozilla.org/firefox/releases/4.0rc1/ .

RC 2 Version

Config Update

  • landed [1] and [2]
  • updated and reconfig pm01

Automated Parts

Some of this release-day work makes sense to do from the release automation. Use force build for these.

Tag

forced build using the webform with values 

your name: jhford
reason for build: retagging for 4.0 release (rc2)
branch to build: <left blank>
revision to build: <left blank>
repository to build: <left blank>
project to build: <left blank>
prop1 name: script_repo_revision
     value: FIREFOX_4_0_RELEASE
prop2 name: release_config
     value: mozilla/release-firefox-mozilla-2.0.py

Source

Clicked "force build"

Major Updates

Both of these done as cltbld@aus2-staging

3.5.17 -> 4.0

Generation
cd /opt/aus2/snippets/staging
# releasetest snippets
rsync --exclude=betatest -av Firefox-3.5.17-build1-4.0rc2-build3-MU-test/ Firefox-3.5.17-build1-4.0-MU-test/
find Firefox-3.5.17-build1-4.0-MU-test/ -type f -exec sed -i -e 's/RC 2//' -e 's/rc2//' {} \;
rsync -av Firefox-3.5.17-build1-4.0rc2-build3-MU-beta/ Firefox-3.5.17-build1-4.0-MU-beta/
find Firefox-3.5.17-build1-4.0-MU-beta/ -type f -exec sed -i -e 's/RC 2//' -e 's/rc2//' {} \;
rsync -av Firefox-3.5.17-build1-4.0rc2-build3-MU/ Firefox-3.5.17-build1-4.0-MU/
find Firefox-3.5.17-build1-4.0-MU/ -type f -exec sed -i -e 's/RC 2//' -e 's/rc2//' {} \;
Quick Verify
# releasetest == beta
find Firefox-3.5.17-build1-4.0-MU-test -type d -iregex '.*/releasetest$' | perl -nle '$a = $_; $a =~ s/releasetest/beta/; $a =~ s/-test/-beta/; system("diff -r -u $_ $a");'
find Firefox-3.5.17-build1-4.0-MU-beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/releasetest/; $a =~ s/-beta/-test/; system("diff -r -u $_ $a");'

# beta == release
find Firefox-3.5.17-build1-4.0-MU -type d -iregex '.*/release$' | perl -nle '$a = $_; $a =~ s/release/beta/; $a =~ s/MU/MU-beta/; system("diff -r -u $_ $a");'
find Firefox-3.5.17-build1-4.0-MU-beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/release/; $a =~ s/-beta//; system("diff -r -u $_ $a");'
Diff against original snippets

Verified that only appv and url have changed with a diff. Saw something like this for all snippets: 

diff -aur Firefox-3.5.17-build1-4.0rc2-build3-MU-test/ Firefox-3.5.17-build1-4.0-MU-test/ | grep -v betatest | grep '^[+-]'
...
-url=http://download.mozilla.org/?product=firefox-4.0rc2-complete&os=win&lang=zh-TW
+url=http://download.mozilla.org/?product=firefox-4.0-complete&os=win&lang=zh-TW
-appv=4.0 RC 2
+appv=4.0
Enable test snippets
~/bin/backupsnip Firefox-3.5.17-build1-4.0-MU-test
~/bin/pushsnip Firefox-3.5.17-build1-4.0-MU-test

3.6.15 -> 3.6

Generation
cd /opt/aus2/snippets/staging
# releasetest snippets
rsync --exclude=betatest -av Firefox-3.6.15-build1-4.0rc2-build3-MU-test/ Firefox-3.6.15-build1-4.0-MU-test/
find Firefox-3.6.15-build1-4.0-MU-test/ -type f -exec sed -i -e 's/RC 2//' -e 's/rc2//' {} \;
rsync -av Firefox-3.6.15-build1-4.0rc2-build3-MU-beta/ Firefox-3.6.15-build1-4.0-MU-beta/
find Firefox-3.6.15-build1-4.0-MU-beta/ -type f -exec sed -i -e 's/RC 2//' -e 's/rc2//' {} \;
rsync -av Firefox-3.6.15-build1-4.0rc2-build3-MU/ Firefox-3.6.15-build1-4.0-MU/
find Firefox-3.6.15-build1-4.0-MU/ -type f -exec sed -i -e 's/RC 2//' -e 's/rc2//' {} \;
Quick Verify
# releasetest == beta
find Firefox-3.6.15-build1-4.0-MU-test -type d -iregex '.*/releasetest$' | perl -nle '$a = $_; $a =~ s/releasetest/beta/; $a =~ s/-test/-beta/; system("diff -r -u $_ $a");'
find Firefox-3.6.15-build1-4.0-MU-beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/releasetest/; $a =~ s/-beta/-test/; system("diff -r -u $_ $a");'

# beta == release
find Firefox-3.6.15-build1-4.0-MU -type d -iregex '.*/release$' | perl -nle '$a = $_; $a =~ s/release/beta/; $a =~ s/MU/MU-beta/; system("diff -r -u $_ $a");'
find Firefox-3.6.15-build1-4.0-MU-beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/release/; $a =~ s/-beta//; system("diff -r -u $_ $a");'
Diff against original snippets

Verified that only appv and url have changed with a diff. Saw something like this for all snippets: 

diff -aur Firefox-3.6.15-build1-4.0rc2-build3-MU-test/ Firefox-3.6.15-build1-4.0-MU-test/ | grep -v betatest | grep '^[+-]'
...
-url=http://download.mozilla.org/?product=firefox-4.0rc2-complete&os=win&lang=zh-TW
+url=http://download.mozilla.org/?product=firefox-4.0-complete&os=win&lang=zh-TW
-appv=4.0 RC 2
+appv=4.0
Enable test snippets
~/bin/backupsnip Firefox-3.6.15-build1-4.0-MU-test
~/bin/pushsnip Firefox-3.6.15-build1-4.0-MU-test

Bouncer Entries

From moz2-linux-slave05 (where the most recent bouncer_submitter run was): 

cd /builds/slave/rel-2.0-bncr-sub/tools/release
wget -O shipped-locales http://hg.mozilla.org/releases/mozilla-2.0/raw-file/FIREFOX_4_0_RELEASE/browser/locales/shipped-locales
PYTHONPATH=../lib/python python tuxedo-add.py --config firefox-tuxedo.ini --product firefox --version 4.0 --tuxedo-server-url https://bounceradmin.mozilla.com/api/ --shipped-locales shipped-locales --bouncer-product-name Firefox --brand-name Firefox --add-mars --milestone 2.0 --platform linux --platform linux64 --platform macosx64 --platform win32 --credentials-file BuildSlaves.py

3.5.18 -> 4.0

3.6.16 -> 3.6

Stage and Rename Files

Summary:

  • Copy files to ~ffxbld and rename to non-RC versions
    • Exclude *checksums* because we don't care for individual locale+platform checksums files in the final release dir
    • Exclude partial MARs because there's no use for them, no point in pushing them to the mirrors
    • Exclude Source packages because they're regenerated for final
    • Exclude *SUMS* because we'll be regenerating them.
    • Exclude all of the stuff that would normally be excluded when going from candidates -> releases (logs, info files, unsigned dir, crsahreporter symbols)

Details: 

# ffxbld@stage
# clean up from the RC1 attempt
rm -rf firefox-4.0
mkdir firefox-4.0
cd firefox-4.0
rsync --exclude=*.log --exclude=*tests* --exclude=*.txt --exclude=*checksums* --exclude='*partial.mar' --exclude=source --exclude=*SUMS* --exclude=unsigned --exclude=*crashreporter* -av /pub/mozilla.org/firefox/nightly/4.0rc2-candidates/build3/ .
rsync -av /pub/mozilla.org/firefox/nightly/4.0-candidates/build1/source/ source/
find contrib* linux* update -type f -exec rename rc2 '' {} \;
find mac win32 -type f -exec rename " RC 2" "" {} \; 
wget -O../firefox-4.0-shipped-locales http://hg.mozilla.org/releases/mozilla-2.0/raw-file/FIREFOX_4_0_RELEASE/browser/locales/shipped-locales
~cltbld/bin/verify-locales.pl -m ~/firefox-4.0-shipped-locales -l bz2
~cltbld/bin/checksum-files .

Then, sign the checksum and source files by doing the following on keymaster: 

mkdir -p ~/signing-work/firefox-4.0
cd ~/signing-work/firefox-4.0
cvs co -d:ext:cltbld@cvs.mozilla.org:/mofo co -d signing release/signing/tools
rsync -av -e 'ssh -i /home/cltsign/.ssh/ffxbld_dsa' ffxbld@stage.mozilla.org:firefox-4.0/*SUMS .
rsync -av -e 'ssh -i /home/cltsign/.ssh/ffxbld_dsa' ffxbld@stage.mozilla.org:firefox-4.0/source/ source/
signing/sign-files .
rsync -av -e 'ssh -i /home/cltsign/.ssh/ffxbld_dsa' *SUMS* ffxbld@stage.mozilla.org:firefox-4.0/
rsync -av -e 'ssh -i /home/cltsign/.ssh/ffxbld_dsa' source/ ffxbld@stage.mozilla.org:firefox-4.0/source/

And then back on stage: 

chown -R ffxbld:firefox linux-i686 linux-x86_64 mac update win32
chmod 644 *SUMS*

Verify locales complains about en-US having XPIs, but that's desired these days, so ignorable.

Other Verifications

Virus Scan

Manually triggered on stage with: 

# ffxbld@stage
python /usr/local/bin/extract_and_run_command.py -j4 clamdscan -m --no-summary -- ~/firefox-4.0

Passed.

Verify permissions

find /home/ffxbld/firefox-4.0/ ! -user ffxbld ! -path \'*/contrib*\'
find /home/ffxbld/firefox-4.0/ ! -group firefox ! -path \'*/contrib*\'
find /home/ffxbld/firefox-4.0/ -type f ! -perm 644
find /home/ffxbld/firefox-4.0/ -mindepth 1 -type d ! -perm 755 ! -path \'*/contrib*\' ! -path \'*/partner-repacks*\'
find /home/ffxbld/firefox-4.0/ -maxdepth 1 -type d ! -perm 2775 -path \'*/contrib*\'

Verify contents against RC

cd ~ffxbld/firefox-4.0
find /pub/mozilla.org/firefox/releases/4.0rc2/ -type f | perl -nle '$a = substr($_,41); $a =~ s/rc2//; $a =~ s/\ RC\ 2//; system("diff","-q",$_,$a);'

Results: 

Files /pub/mozilla.org/firefox/releases/4.0rc2/source/firefox-4.0rc2.bundle.asc and source/firefox-4.0.bundle.asc differ
Files /pub/mozilla.org/firefox/releases/4.0rc2/source/firefox-4.0rc2.bundle and source/firefox-4.0.bundle differ
Files /pub/mozilla.org/firefox/releases/4.0rc2/source/firefox-4.0rc2.source.tar.bz2 and source/firefox-4.0.source.tar.bz2 differ
Files /pub/mozilla.org/firefox/releases/4.0rc2/source/firefox-4.0rc2.source.tar.bz2.asc and source/firefox-4.0.source.tar.bz2.asc differ

diff: update/linux-i686/af/firefox-4.0rc1-4.0.partial.mar: No such file or directory
diff: update/linux-i686/ak/firefox-4.0rc1-4.0.partial.mar: No such file or directory
...
diff: update/win32/zh-CN/firefox-4.0rc1-4.0.partial.mar: No such file or directory
diff: update/win32/zh-TW/firefox-4.0rc1-4.0.partial.mar: No such file or directory
diff: update/win32/zu/firefox-4.0rc1-4.0.partial.mar: No such file or directory
Files /pub/mozilla.org/firefox/releases/4.0rc2/MD5SUMS and MD5SUMS differ
Files /pub/mozilla.org/firefox/releases/4.0rc2/MD5SUMS.asc and MD5SUMS.asc differ
Files /pub/mozilla.org/firefox/releases/4.0rc2/SHA1SUMS and SHA1SUMS differ
Files /pub/mozilla.org/firefox/releases/4.0rc2/SHA1SUMS.asc and SHA1SUMS.asc differ

All ignorable:

  • Source files differ because we regenerated them
  • Partials are missing because we purposely don't include them
  • SUMS files differ because we regenerated them

Add Index Files

index.html was e-mailed to me by LegNeato, and I put it in ~ffxbld@stage 

# ffxbld@stage
cd ~/firefox-4.0
cp ~/index.html .
for dir in `find . -type d -mindepth 1`; do cp -pv index.html $dir/; done

Push index files

justdave says that we should push the index.html 6 hours before the rest of the contents, to ensure it gets everywhere ahead of everything else. Use this command to do so: 

rsync -av --exclude=KEY --exclude=*.xpi --exclude=*.bz2 --exclude=*.dmg --exclude=*.exe --exclude=*.mar --exclude=*.bundle --exclude=*SUMS* --exclude=*.asc /home/ffxbld/firefox-4.0/ /pub/mozilla.org/firefox/releases/4.0/

Push to Mirrors

rsync -av /home/ffxbld/firefox-4.0/ /pub/mozilla.org/firefox/releases/4.0/

Update rsync modules

Removed Firefox 3.5.17 from mozilla-current and added 4.0, in stage:/pub/mozilla.org/pub/zz/rsyncd-mozilla-current.exclude. We still have 3.5.17 in mozilla-releases and get plenty of coverage there for the set of people still on 3.5.x.

Notify AV Vendors

Sent notification to AV vendors after the push to mirrors, so that they can scan the files.

Repush MU and notify release-drivers of MU testing

3.5.18 and 3.6.16 overwrote the major updates on releasetest so repushed them: 

~/bin/backupsnip Firefox-3.5.17-build1-4.0-MU-test
~/bin/pushsnip Firefox-3.5.17-build1-4.0-MU-test

~/bin/backupsnip Firefox-3.6.15-build1-4.0-MU-test
~/bin/pushsnip Firefox-3.6.15-build1-4.0-MU-test

E-mailed release-drivers and let them know that both MUs are ready to be tested on releasetest when uptake over 10k.

Push Major Updates

NOT DONE YET
Dirs to push:

  • Firefox-3.5.17-build1-4.0rc2-build3-MU-beta
  • Firefox-3.5.17-build1-4.0rc2-build3-MU
  • Firefox-3.6.15-build1-4.0rc2-build3-MU
  • Firefox-3.6.15-build1-4.0rc2-build3-MU-beta

Remove index files

NOT DONE YET ffxbld@stage 

cd /pub/mozilla.org/firefox/releases/4.0
find . -name index.html -exec rm -v {} \;

Update symlinks on FTP

NOT DONE YET 

ln -s 4.0 latest-4.0  
ln -s latest-4.0 latest

XULRunner

No need to stage or rename files because the 2.0rc1 versions already use "2.0".

Push to Mirrors

Update Wiki Pages

Retrieved from "https://wiki.allizom.org/index.php?title=Releases/Firefox_4.0/BuildNotes&oldid=293035"