Releases/Thunderbird 10.0.5esr/BuildNotes
Notes About Releasing
Please update the Notes Template and the Release:Primer for future releases (bug fixes, changes to automation) as needed
Bugs hit
Enter any bugs pre-existing or newly discovered and filed during the release:
- bug 760353 - make source-package fails on comm-esr10 because MKDIR is undefined
- bug 760361 - l10n repacks failing in configure
Build Engineers
nthomas - Tracking bug: bug 744599
Signed-off Revision(s)
- comm-esr10: https://hg.mozilla.org/releases/comm-esr10/rev/570c22432c13
- mozilla-esr10: http://hg.mozilla.org/releases/mozilla-esr10/rev/5713c92407dd
- L10N changesets: unchanged
Build data
Firefox
Build # | Type | Build ID | Build machine | Time to build |
1 | Linux | |||
Linux64 | ||||
Mac | ||||
Windows |
Notes
Build 1
Preparing to start Automation
- first TB ESR release on Firefox infra so nothing to clobber
- Reserve slaves (you need ~5 each of linux, linux64, win32 (w32), and macosx64 (moz2-darwin10)). Doing this 2-3 hours in advance should speed things up. e.g.:
[cltbld@buildbot-master34 master]$ pwd /builds/buildbot/build1/master [cltbld@buildbot-master43 master]$ cat reserved_slaves 6
- For build1 only, create a symlink for your $version (eg 11.0b4, 10.0.2esr, 10.0.2, 3.6.27) (until bug 725839 is resolved)
# tbirdbld@stage cd /pub/mozilla.org/thunderbird/nightly/ version=10.0.5esr mkdir ../candidates/$version-candidates ln -s ../candidates/$version-candidates $version-candidates
- No l10n changes
- Double land automation configs
- Tag buildbot-configs, buildbotcustom, & tools with build & release tags. (details)
hg tag -f THUNDERBIRD_10_0_5esr_{RELEASE,BUILD1}
- update and reconfigure the master
- dry run automation start
cd /builds/buildbot/build1/master source ../bin/activate PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u nthomas -V 10.0.5esr \ --branch comm-esr10 --build-number 1 \ --release-config release-thunderbird-comm-esr10.py --products thunderbird \ --dryrun -l localhost:9001
- start automation ← monitor progress on buildbot (e.g. comm-esr10)
- the same command as above but without the "--dryrun"
E-mail Metrics
Use the address "metrics-alerts < AT > mozilla < PERIOD > org". Note for first-time-releasers: your email will get held for moderator approval - that is expected. If it happens more than once, get help on #metrics.
Thunderbird,thunderbird,thunderbird,10.0.5,10.0 Thunderbird,thunderbird,thunderbird,10.0.5esr,10.0
E-mail mirrors@mozilla.org
N/A
Edit rsync exclude files
N/A
Tag
No problems
Bouncer Submitter
No problems
Source
Failed:
make source-package hg-bundle HG_BUNDLE_REVISION=7c765c397d25 in dir /builds/slave/tb-rel-comm-esr10-thunderbird-source/comm-esr10/objdir-tb (timeout 2700 secs) make[1]: Entering directory `/builds/slave/tb-rel-comm-esr10-thunderbird-source/comm-esr10/objdir-tb/mail/installer' Packaging source tarball... p ../../mozilla/dist/source/ make[1]: p: Command not found make[1]: [source-package] Error 127 (ignored) (cd /builds/slave/tb-rel-comm-esr10-thunderbird-source/comm-esr10 && gtar -c --owner=0 --group=0 --numeric-owner --mode="go-w" --exclude=".hg*" --exclude="CVS" --exclude=".cvs*" --exclude=".mozconfig*" --exclude="*.pyc" --exclude="/builds/slave/tb-rel-comm-esr10-thunderbird-source/comm-esr10/mozilla/Makefile" --exclude="/builds/slave/tb-rel-comm-esr10-thunderbird-source/comm-esr10/mozilla/dist" --exclude="objdir-tb" -f - ../comm-esr10) | bzip2 -vf > ../../mozilla/dist/source/thunderbird-10.0.5esr.source.tar.bz2 /bin/sh: ../../mozilla/dist/source/thunderbird-10.0.5esr.source.tar.bz2: No such file or directory gtar: Removing leading `../' from member names make[1]: *** [source-package] Error 1 make[1]: Leaving directory `/builds/slave/tb-rel-comm-esr10-thunderbird-source/comm-esr10/objdir-tb/mail/installer' make: *** [source-package] Error 2
bug 760353 filed.
DONE TO HERE
Build
en-US
linux32, linux64 and win32 builds ok, mac pending
Repacks
Failing in configure, bug 760361 filed on:
/builds/slave/tb-rel-comm-esr10-lnx64-rpk-1/comm-esr10/configure Adding configure options from /builds/slave/tb-rel-comm-esr10-lnx64-rpk-1/comm-esr10/.mozconfig: --with-l10n-base=/builds/slave/tb-rel-comm-esr10-lnx64-rpk-1/comm-esr10/obj-l10n/../../mozilla-esr10 --enable-application=mail --enable-official-branding --enable-update-channel=esr --enable-update-packaging creating cache ./config.cache checking host system type... x86_64-unknown-linux-gnu checking target system type... x86_64-unknown-linux-gnu checking build system type... x86_64-unknown-linux-gnu checking for gawk... gawk configure: error: Invalid value --with-l10n-base, /builds/slave/tb-rel-comm-esr10-lnx64-rpk-1/comm-esr10/obj-l10n/../../mozilla-esr10 doesn't exist
NOTES TRIMMED TO HERE
L10N verify
Checksums
Updates
Update verify
Major Update
Major update generation is not kicked off automatically. You should only create them once the final builds for the "latest" build are ready. For instance, the 3.6.26 builds got created a week before the 10.0 release but the "final" 10.0 builds were not created after few days.
Follow instructions from the Major update documentation.
Major Update verify
Reset reserved slaves
Check permissions / AV scan
It is supposed to be triggered automatically after updates are done. File a bug if it doesn't. details
Push to internal mirrors
This is done by automation for rapid release betas. Note: if you have to reconfig the release buildbot master in during the release, the uptake monitoring will fail (bug 629648). That means you'll need to manually send the "ready for releasetest" emails when you believe things are "good enough". Refer to a prior release for email details.
For other releases, this should be done manually at this point, after check permissions / AV scan have completed.
- Make sure the rsync excludes have been updated (Releases/BuildNotesTemplate#Edit_rsync_exclude_files)
- Force build the push to mirrors builder. Make sure at least 15 minutes have elapsed since editing the rysnc module to give it time to sync out.
Final verification
Record receipt of "completed_final_verification" emails (1 per platform)
Going to Beta
NOTE: Skip to "Before Release Day" for beta releases
Run backupsnip
NOTE: Remember to do this at least 2 hours ahead of the expected "go to beta" email. details
Remember to ssh to aus3-staging using your short LDAP name and then run sudo su - ffxbld
once on. If you intend to use screen, initiate your screen session while you are still yourself.
Push to beta
Wait for "go" from release managers
For Firefox:
- Publish (and record run time here): details
- on completion, email r-d that VERSION updates are live on the beta channel
For Android:
- publish on Google Play (nee Android Marketplace), using instructions below
- on completions, email r-d that VERSION is live on Google Play
Wait for confirmation from QA that each of the above releases is okay, then close the tracking ticket.
Before Release Day
Assuming this is not a chemspill.
Run backupsnip for release snippets
Do this the night before release day as it can take up to two hours.
Push index file to mirrors
Push to external mirrors
If this is a chemspill you will want to ping justdave (or other sysadmin) in #release-drivers and ask them to enable CDN.
- Remove previously added blacklisting entry from stage.mozilla.org:/pub/mozilla.org/zz/rsyncd-mozilla-releases.exclude
- Replace old version with the current one in stage.mozilla.org:/pub/mozilla.org/zz/rsyncd-mozilla-current.exclude
See details
Release Day
Publish Fennec to the Play Store (for beta releases and release releases)
Push the files (for beta releases and release releases)
- Use this script and run it as ffxbld@stage (please push you changes prior to using it, and grab latest on stage via curl -O http://hg.mozilla.org/build/braindump/raw-file/tip/releases-related/push_fennec.sh)
Play Store (for beta releases and release releases)
IMPORTANT - Make sure to follow instructions!
- download multi apk (e.g. multi/fennec-10.0b6.multi.android-arm.apk)
- visit https://market.android.com/publish
- choose "Firefox Beta" or "Firefox"
- select the "APK Files" tab and choose "Upload APK"
- choose the multi apk that you downloaded and hit "upload"
- NOTE: the progress bar does not seem to show progress (at least on Google Chrome)
- when it finishes uploading, verify VersionCode is the build date (almost anyway, the market reports an hour later than our buildID)
- hit "Save"
- choose the multi apk that you downloaded and hit "upload"
- activate new apk (You should see an Error message since both apk's are active)
- deactivate old apk (You should not see the Error message anymore)
- hit "Save"
- If this is a beta 1, go to "Product Details" tab -> Recent Changes, change the url to http://www.mozilla.com/en-US/mobile/{VERSION}beta/releasenotes/
- and hit "Save"
Push XULRunner to Mirrors
Update XULRunner wiki page
For major releases or chemspills, update the links on:
Also for older releases:
- https://developer.mozilla.org/en/XULRunner_1.9.2_Release_Notes
- https://developer.mozilla.org/en/Gecko_SDK
Check Throttling
See http://update-watch.localgho.st/release/ for example AUS links
Some links to check:
- Automatic (idle time check) update links:
- Manual update links:
Push to Release Channel (for beta releases and release releases)
Once there is enough uptake and we get "go" from release driver.
Update symlinks
Once we're signed off on the release channel.
Remove index.html files
If you created them earlier to hide the release.