Confirmed users
4,293
edits
No edit summary |
|||
(30 intermediate revisions by 2 users not shown) | |||
Line 4: | Line 4: | ||
= Bugs hit = | = Bugs hit = | ||
Enter any bugs pre-existing or newly discovered and filed during the release: | Enter any bugs pre-existing or newly discovered and filed during the release: | ||
* {{bug|}} - | * {{bug|760353}} - make source-package fails on comm-esr10 because MKDIR is undefined | ||
* | * {{bug|760361}} - l10n repacks failing in configure | ||
* <strike>{{bug|760385}} - we clobber hg shares more frequently than necessary</strike> dupe of {{bug|748158}} | |||
* {{bug|760389}} - some builders not clobbered on build2 | |||
* {{bug|760390}} - en-US builds not pulling right mozilla-esr10 revision | |||
* {{bug|760397}} - source builder not getting right mozilla-esr10 revision | |||
* {{bug|760807}} - Program Compatibility Assistant shown on Windows 7 when setting as default mail client (NSIS version) | |||
* {{bug|761951}} - not checking releasetest in final verification | |||
* {{bug|763769}} - release sanity complains about mozconfig differences | |||
=Build Engineers= | =Build Engineers= | ||
Line 11: | Line 18: | ||
=Signed-off Revision(s)= | =Signed-off Revision(s)= | ||
* comm-esr10: https://hg.mozilla.org/releases/comm-esr10/rev/570c22432c13 | |||
* mozilla-esr10: http://hg.mozilla.org/releases/mozilla-esr10/rev/5713c92407dd | L10N changesets: unchanged | ||
* | |||
Build1: | |||
* comm-esr10: https://hg.mozilla.org/releases/comm-esr10/rev/570c22432c13 on default | |||
* mozilla-esr10: http://hg.mozilla.org/releases/mozilla-esr10/rev/5713c92407dd on GECKO1005_2012053118_RELBRANCH | |||
Build2: | |||
* comm-esr10: https://hg.mozilla.org/releases/comm-esr10/rev/48677c4f453a on GECKO1005_2012053119_RELBRANCH | |||
* mozilla-esr10: http://hg.mozilla.org/releases/mozilla-esr10/rev/5713c92407dd on GECKO1005_2012053118_RELBRANCH | |||
Build 3: | |||
* comm-esr10: https://hg.mozilla.org/releases/comm-esr10/rev/939269a86b88 on GECKO1005_2012053119_RELBRANCH | |||
* mozilla-esr10: https://hg.mozilla.org/releases/mozilla-esr10/rev/5713c92407dd on GECKO1005_2012053118_RELBRANCH | |||
=Build data= | =Build data= | ||
Line 26: | Line 44: | ||
| rowspan="4" | 1 | | rowspan="4" | 1 | ||
| Linux | | Linux | ||
| rowspan="4" | | | rowspan="4" | 20120531193048 | ||
| | | mv-moz2-linux-ix-slave09 | ||
| | | 38 mins, 47 secs | ||
|- | |||
| Linux64 | |||
| bld-centos5-64-vmw-003 | |||
| 45 mins, 40 secs | |||
|- | |||
| Mac | |||
| moz2-darwin10-slave44 | |||
| 3 hrs, 32 mins, 31 secs | |||
|- | |||
| Windows | |||
| mw32-ix-slave26 | |||
| 1 hrs, 43 mins, 28 secs | |||
|- | |||
| rowspan="1" | 2 | |||
| Linux | |||
| rowspan="1" | 20120601013708 | |||
| mv-moz2-linux-ix-slave09 | |||
| 38 mins, 13 secs | |||
|- | |||
| rowspan="4" | 3 | |||
| Linux | |||
| rowspan="4" | 20120601041309 | |||
| mv-moz2-linux-ix-slave09 | |||
| 37 mins, 26 secs | |||
|- | |- | ||
| Linux64 | | Linux64 | ||
| | | bld-centos5-64-vmw-003 | ||
| | | 41 mins, 53 secs | ||
|- | |- | ||
| Mac | | Mac | ||
| | | moz2-darwin10-slave44 | ||
| | | 3 hrs, 32 mins, 5 secs | ||
|- | |- | ||
| Windows | | Windows | ||
| | | mw32-ix-slave02 | ||
| | | 1 hrs, 39 mins, 32 secs | ||
|- | |||
| 3 Spin 2 | |||
| Windows | |||
| 20120605154056 | |||
| mw32-ix-slave02 | |||
| 1 hrs, 39 mins, 18 secs | |||
|} | |} | ||
Line 112: | Line 160: | ||
make: *** [source-package] Error 2 | make: *** [source-package] Error 2 | ||
</pre> | </pre> | ||
{{Bug|760353}} filed. | |||
=== Build === | |||
==== en-US ==== | |||
No problems. | |||
=== | ==== Repacks ==== | ||
==== | Failing in configure, {{bug|760361}} filed on: | ||
<pre>/builds/slave/tb-rel-comm-esr10-lnx64-rpk-1/comm-esr10/configure | |||
Adding configure options from /builds/slave/tb-rel-comm-esr10-lnx64-rpk-1/comm-esr10/.mozconfig: | |||
--with-l10n-base=/builds/slave/tb-rel-comm-esr10-lnx64-rpk-1/comm-esr10/obj-l10n/../../mozilla-esr10 | |||
--enable-application=mail | |||
--enable-official-branding | |||
--enable-update-channel=esr | |||
--enable-update-packaging | |||
creating cache ./config.cache | |||
checking host system type... x86_64-unknown-linux-gnu | |||
checking target system type... x86_64-unknown-linux-gnu | |||
checking build system type... x86_64-unknown-linux-gnu | |||
checking for gawk... gawk | |||
configure: error: Invalid value --with-l10n-base, /builds/slave/tb-rel-comm-esr10-lnx64-rpk-1/comm-esr10/obj-l10n/../../mozilla-esr10 doesn't exist | |||
</pre> | |||
Standard8 ported some build changes over to comm-esr10, and we're respinning. | |||
===== | ==Build 2== | ||
===Preparing to start Automation=== | |||
<small>[[Release:Release_Automation_on_Mercurial:Documentation#Starting_the_automation|detailed instructions]]</small> | |||
* clobber Any master, release-comm-esr10 branch, any builder | |||
* Double land [http://hg.mozilla.org/build/buildbot-configs/rev/d63bb0b59645 config bump] and [http://hg.mozilla.org/build/buildbot-configs/rev/d63bb0b59645 l10n mozconfig fix]. | |||
* Retag buildbot-configs, buildbotcustom, & tools with build & release tags. <small>''([[Release:Release_Automation_on_Mercurial:Documentation#Starting_the_automation|details]])''</small> | |||
** buildbot-configs: hg tag -f THUNDERBIRD_10_0_5esr_{RELEASE,BUILD2} | |||
** buildbotcustom and tools, on build1 revision: hg tag THUNDERBIRD_10_0_5esr_BUILD2 | |||
* update and reconfigure the master | |||
* dry run automation start | |||
<pre> | |||
cd /builds/buildbot/build1/master | |||
source ../bin/activate | |||
PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u nthomas -V 10.0.5esr \ | |||
--branch comm-esr10 --build-number 2 \ | |||
--release-config release-thunderbird-comm-esr10.py --products thunderbird \ | |||
--dryrun -l localhost:9001 | |||
</pre> | |||
* start automation ''← monitor progress on buildbot (e.g. [http://buildbot-master34.build.mozilla.org:8001/waterfall?num_events=10&category=release-comm-esr10- comm-esr10])'' | |||
** the same command as above but without the "--dryrun" | |||
=== | === E-mail Metrics === | ||
N/A for build 2 | |||
=== | === E-mail mirrors@mozilla.org === | ||
N/A | |||
=== | === Edit rsync exclude files === | ||
N/A | |||
=== | === Tag === | ||
No problems. | |||
=== Bouncer Submitter === | |||
N/A | |||
=== Source === | |||
Failed to clobber, {{bug|760389}}. Manually clobbered and rebuilt, green. | |||
Standard8 discovered it had the wrong mozilla-esr10 code - {{bug|760397}}. | |||
=== Build === | |||
Also didn't clobber. Standard8 discovered that they have a gecko version of 10.0.6esrpre, so are getting source from mozilla-esr10's default branch - {{bug|760390}} | |||
Another respin required. | |||
=== | ==Build 3== | ||
===Preparing to start Automation=== | |||
<small>[[Release:Release_Automation_on_Mercurial:Documentation#Starting_the_automation|detailed instructions]]</small> | |||
* clobber Any master, release-comm-esr10 branch, any builder | |||
* do manual clobbers of source, *_build, *repack* | |||
* Double land [http://hg.mozilla.org/build/buildbotcustom/rev/83fd5c0b5818 source builder fix] and [http://hg.mozilla.org/build/buildbot-configs/rev/0aebc3c5abcc config bump] | |||
* Standard8 landed the [http://hg.mozilla.org/releases/comm-esr10/rev/939269a86b88 mozconfig fix] | |||
* Retag buildbot-configs, buildbotcustom, & tools with build & release tags <small>''([[Release:Release_Automation_on_Mercurial:Documentation#Starting_the_automation|details]])''</small> | |||
** buildbotcustom & buildbot-configs: hg tag -f THUNDERBIRD_10_0_5esr_{RELEASE,BUILD3} | |||
** tools, on build2 revision: hg tag -r 82e252ae7d68 THUNDERBIRD_10_0_5esr_BUILD3 | |||
* update and reconfigure the master | |||
* dry run automation start | |||
<pre> | |||
cd /builds/buildbot/build1/master | |||
source ../bin/activate | |||
PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u nthomas -V 10.0.5esr \ | |||
--branch comm-esr10 --build-number 3 \ | |||
--release-config release-thunderbird-comm-esr10.py --products thunderbird \ | |||
--dryrun -l localhost:9001 | |||
2012-06-01 03:51:57,136 : INFO : Branch comm-esr10 is enabled on master for thunderbird | |||
2012-06-01 03:51:57,636 : INFO : Comparing thunderbird mozconfigs to nightly mozconfigs... | |||
2012-06-01 03:51:58,037 : ERROR : found in mail/config/mozconfigs/macosx-universal/release but not in mail/config/mozconfigs/macosx-universal/nightly: mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print --comm-rev=${COMM_REV} --mozilla-rev=${MOZILLA_REV}" | |||
2012-06-01 03:51:58,037 : WARNING : mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print" not in macosx64 ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --with-macbundlename-prefix=Firefox', 'ac_add_options --enable-codesighs', 'mk_add_options MOZ_MAKE_FLAGS="-j4"', 'ac_add_options --enable-signmar', 'ac_add_options --enable-js-diagnostics', 'ac_add_options --disable-install-strip']! | |||
2012-06-01 03:51:58,038 : ERROR : found in mail/config/mozconfigs/macosx-universal/nightly but not in mail/config/mozconfigs/macosx-universal/release: mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print" | |||
2012-06-01 03:51:58,469 : ERROR : found in mail/config/mozconfigs/win32/release but not in mail/config/mozconfigs/win32/nightly: mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print --comm-rev=${COMM_REV} --mozilla-rev=${MOZILLA_REV}" | |||
2012-06-01 03:51:58,469 : WARNING : mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print" not in win32 ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --enable-signmar', 'ac_add_options --enable-js-diagnostics', '. $topsrcdir/configs/mozilla2/win32/include/choose-make-flags', 'mk_add_options MOZ_MAKE_FLAGS=-j1']! | |||
2012-06-01 03:51:58,470 : ERROR : found in mail/config/mozconfigs/win32/nightly but not in mail/config/mozconfigs/win32/release: mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print" | |||
2012-06-01 03:51:58,754 : ERROR : found in mail/config/mozconfigs/linux64/release but not in mail/config/mozconfigs/linux64/nightly: mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print --comm-rev=${COMM_REV} --mozilla-rev=${MOZILLA_REV}" | |||
2012-06-01 03:51:58,754 : WARNING : mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print" not in linux64 ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --enable-codesighs', 'mk_add_options MOZ_MAKE_FLAGS="-j4"', 'ac_add_options --enable-signmar', 'ac_add_options --enable-js-diagnostics', 'ac_add_options --with-ccache=/usr/bin/ccache', 'export MOZILLA_OFFICIAL=1', 'export MOZ_TELEMETRY_REPORTING=1', "mk_add_options PROFILE_GEN_SCRIPT='$(PYTHON) @MOZ_OBJDIR@/_profile/pgo/profileserver.py 10'"]! | |||
2012-06-01 03:51:58,754 : ERROR : found in mail/config/mozconfigs/linux64/nightly but not in mail/config/mozconfigs/linux64/release: mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print" | |||
2012-06-01 03:51:59,058 : ERROR : found in mail/config/mozconfigs/linux32/release but not in mail/config/mozconfigs/linux32/nightly: mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print --comm-rev=${COMM_REV} --mozilla-rev=${MOZILLA_REV}" | |||
2012-06-01 03:51:59,059 : WARNING : mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print" not in linux ['ac_add_options --enable-update-channel=nightly', 'CXX=$REAL_CXX', 'CXX="ccache $REAL_CXX"', 'CC="ccache $REAL_CC"', 'mk_add_options PROFILE_GEN_SCRIPT=@TOPSRCDIR@/build/profile_pageloader.pl', 'ac_add_options --enable-codesighs', 'mk_add_options MOZ_MAKE_FLAGS="-j4"', 'ac_add_options --enable-signmar', 'ac_add_options --enable-js-diagnostics', 'ac_add_options --with-ccache=/usr/bin/ccache', 'export MOZILLA_OFFICIAL=1', 'export MOZ_TELEMETRY_REPORTING=1', "mk_add_options PROFILE_GEN_SCRIPT='$(PYTHON) @MOZ_OBJDIR@/_profile/pgo/profileserver.py 10'"]! | |||
2012-06-01 03:51:59,059 : ERROR : found in mail/config/mozconfigs/linux32/nightly but not in mail/config/mozconfigs/linux32/release: mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print" | |||
2012-06-01 03:51:59,059 : ERROR : Error verifying mozconfigs | |||
</pre> | |||
Which is fallout from the mozconfig change | |||
* start automation wth -b to bypass error ''← monitor progress on buildbot (e.g. [http://buildbot-master34.build.mozilla.org:8001/waterfall?num_events=10&category=release-comm-esr10- comm-esr10])'' | |||
** the same command as above but without the "--dryrun" | |||
=== E-mail Metrics === | |||
=== | N/A for build 3 | ||
=== E-mail mirrors@mozilla.org === | |||
N/A | |||
=== | === Edit rsync exclude files === | ||
N/A | |||
=== | === Tag === | ||
No problems | |||
=== Bouncer Submitter === | |||
N/A | |||
=== Source === | |||
No problems | |||
==== | === Build === | ||
==== en-US ==== | |||
* linux32 ok | |||
* linux64 and Mac failed in the compile step: | |||
<pre>abort: unknown revision 'THUNDERBIRD_10_0_5esr_RELEASE'! | |||
Time: real 0.900 secs (user 0.467+0.000 sys 0.100+0.000) | |||
Traceback (most recent call last): | |||
File "/builds/slave/tb-rel-comm-esr10-osx64-bld/build/client.py", line 618, in <module> | |||
Executing command: ['python', '../tools/buildfarm/utils/hgtool.py', 'http://hg.mozilla.org/releases/mozilla-esr10/', '/builds/slave/tb-rel-comm-esr10-osx64-bld/build/mozilla'] | |||
Executing command: ['hg', 'update', '-r', 'THUNDERBIRD_10_0_5esr_RELEASE', '-R', '/builds/slave/tb-rel-comm-esr10-osx64-bld/build/mozilla', '--verbose', '--time'] | |||
</pre> | |||
which is related to {{bug|757798}}. Solution is to manually pull in /builds/hg-shared/releases/mozilla-esr10 on bld-centos5-64-vmw-003 and moz2-darwin10-slave44, then clobber the build dirs. Also do the pull on other slaves that have repo to avoid later (potential) bustage in l10n repacks). The rebuild linux64 and mac en-US (green) | |||
* win32 failed on the same share issue; updated the share & manual clobber, then rebuild (green) | |||
=== | ==== l10n repacks ==== | ||
No problems | |||
=== Checksums === | |||
No problems | |||
==== | === Updates === | ||
No problems | |||
==== Update verify ==== | |||
No problems | |||
=== Reset reserved slaves === | === Reset reserved slaves === | ||
No done, other releases using machines/master. | |||
=== Check permissions / AV scan === | === Check permissions / AV scan === | ||
No problems | |||
=== Push to internal mirrors === | === Push files to internal mirrors === | ||
No problems | |||
=== Final verification === | === Final verification === | ||
No problems | |||
'''{{Bug|760807}} will means we need to redo win32''' | |||
==== | == Build 3 Spin 2 == | ||
===Preparing to start Automation=== | |||
* rather than update the release config, move the tags etc etc etc when we are rebuilding windows only we do this file munging on ftp (hard linking): | |||
# tbirdbld@stage | |||
/pub/mozilla.org/thunderbird/candidates/10.0.5esr-candidates | |||
cp -alv build3{,.pre-win32-rebuild}/ | |||
cd build3 | |||
rm -rf jsshell-win32.zip *SUMS* update/win32/ win32/ win32_info.txt | |||
ie hard link as a quick backup, then remove win32 specific files and SUMS files that will need be regenerated | |||
* manual clobber of tb-rel-comm-esr10* on mw32-ix-slave02 and 03 | |||
=== E-mail Metrics === | |||
N/A | |||
=== | === E-mail mirrors@mozilla.org === | ||
N/A | |||
=== Edit rsync exclude files === | |||
N/A | |||
=== Tag === | |||
N/A | |||
=== Bouncer Submitter === | |||
N/A | |||
=== Source === | |||
N/A | |||
=== | === Build === | ||
==== en-US ==== | |||
Used the rebuild button, no problems | |||
==== l10n repacks ==== | |||
mw32-ix-slave10 didn't have makensis.exe renamed to makensisu-2.46.exe, so repack 6 was using 2.33 when configure ran. Job aborted, clobbered, nsis fixed, rebooted; sr removed from candidate dir; builder rebuilt. | |||
All green after that. | |||
=== | === Checksums === | ||
Rebuilt. Failed at the very end where it tried to recreate the contrib dirs which already existed. Harmless so ignored. | |||
=== | === Updates === | ||
Backed out update verify configs, and rebuilt. No problems. | |||
==== Update verify ==== | |||
No problems | |||
=== Check permissions / AV scan === | |||
Both rebuilt. Permissions orange with: | |||
WARN: target directory /pub/mozilla.org/thunderbird/releases/10.0.5esr/ exists | |||
which is fine because that dir exists but is empty after emptying the earlier builds out. | |||
A/V no problems. | |||
=== | === Push files to internal mirrors === | ||
Failed because thunderbird/releases/10.0.5esr already existed. Removed this and rebuilt. Twice it turns out, but the first one succeeded ok. | |||
=== | === Final verification === | ||
All green, but not actually checking the releasetest channel, so not verifying bouncer links OK - {{bug|761951}}. While writing the patch for that tested that win32 works OK on releasetest. | |||
=== | === Backup snippets === | ||
time ~/bin/backupsnip Thunderbird-10.0.5esr-build3 | |||
real 0m37.623s | |||
=== Reset reserved slaves === | |||
In this case it would be about evacuating the slaves from bm34, which is limited to T'bird releases again, but we'll need them for 14.0b1 anyway. | |||
=== | === Push to esr channel === | ||
time ~/bin/pushsnip Thunderbird-10.0.5esr-build3 | |||
=== Update symlinks === | |||
<pre> | |||
#tbirdbld@stage | |||
cd /pub/mozilla.org/thunderbird/releases | |||
rm latest-10.0esr; ln -s 10.0.5esr latest-10.0esr | |||
</pre> | |||