ReleaseEngineering/Maintenance: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
Line 23: Line 23:
| style="background:#cccccc" | '''When'''
| style="background:#cccccc" | '''When'''
| style="background:#cccccc" | '''Bug #(s)''' - '''Description(s)'''
| style="background:#cccccc" | '''Bug #(s)''' - '''Description(s)'''
|-
| in production
| 2012-04-10 1100 PST
|
* {{bug|744049}} - tcheckerboard always reports 1.0 (tegra talos web server updated to talos tip)
|-
|-
| in production
| in production

Revision as of 18:59, 10 April 2012

This page is to track upcoming changes to any part of RelEng infrastructure; buildbot masters, slaves, ESX hosts, etc. This should allow us keep track of what we're doing in a downtime, and also what changes can be rolled out to production without needing a downtime. This should be helpful if we need to track what changes were made when troubleshooting problems.

ReleaseEngineering:BuildbotBestPractices describes how we manage changes to our masters.

Relevant repositories

Are you changing the tool chain on a master? If so, let auto-tools know so they can update their masters

Reconfigs / Deployments

This page is updated by the person who does a reconfig on production systems. Please give accurate times, as we use this page to track down if reconfigs caused debug intermittent problems.

Did you change the tool chain on a master? If so, let auto-tools know so they can update their masters

Outcome should be 'backed out' or 'In production' or some such. Reverse date order pretty please.

Outcome When Bug #(s) - Description(s)
in production 2012-04-10 1100 PST
  • bug 744049 - tcheckerboard always reports 1.0 (tegra talos web server updated to talos tip)
in production 2012-04-09 0700 PDT
  • bug 607392 - split tagging into en-US and other
  • bug 721885 - shut off unused branch
  • bug 400296 - Have release automation support signing OSX builds (up to 10.7 support)
in production 2012-04-04 11:00 PDT
  • bug 690311 - deploy newer version of cleanup.py to the foopies
in production 2012-03-30 6:15 PDT
  • bug 738588 - add ts_paint to the android tests.
  • bug 737458 - replace tpr_responsiveness for tp5row.
  • bug 737458 - add tpr_responsiveness temporarily for mozilla-central and larch.
  • bug 740599 - update staging release config files;
in production 2012-03-29 6:55 PDT
  • bug 715193 - If a branch does not use talos_from_source_code we should fallback to talos.mobile.old.zip (fixes esr10).
in production 2012-03-28 16:35 PDT
  • bug 740196 - ts_paint on Android doesn't actually work
in production 2012-03-28 11:55 PDT
  • bug 737632 - Remove jaegermonkey, graphics and pine to reduce builders
  • bug 723667 - fix Android trobocheck and ts_paint tests.
  • bug 739486 - test-masters.sh should run ./setup_master.py -t
  • add option to setup_master.py to print error logs when hit
  • bug 723667 - enable trobopan and tcheckerboard by default (not for m-a/m-b/m-r/1.9.2)
  • bug 627182 - Automate signing and publishing of XULRunner builds. r=bhearsum
in production 2012-03-27 12:30 PDT
  • bug 723667 - Add trobopan and trobocheck to m-c/m-i. r=jmaher
in production 2012-03-23 11:30 PDT
in production 2012-03-27 01:35 PDT
in production 2012-03-23 7:00 PDT
  • bug 737864 - Tweak release category for Thunderbird.
  • bug 737458 - add tp5row side by side and cleanup config.py.
  • bug 737581 - enable peptest on m-c and m-i.
  • bug 713846 - Treat 'fennec' builds as having product 'mobile' for the purposes of uploading logs.
backout 2012-03-21 11:45 PDT
  • bug 737427. Use 1024x768 as the screen resolution for the tegras.
in production 2012-03-21 9:30 PDT
  • bug 737427. Use 1024x768 as the screen resolution for the tegras.
  • bug 697150. (Bv1) Remove 'ac_add_options --disable-installer' for XulRunner current branches.
  • bug 733394. Add leak test logic to mozilla-beta.
  • bug 736587. Enable Android for pine.
in production 2012-03-20 8:30 PDT
in production 2012-03-19 10:30 PDT
in production 2012-03-16 8:45 PDT
  • bug 734996 - fennec beta release update channel -> beta.
  • bug 734221 - deploy updateSUT.py and upgrade the boards to SUT Agent version 1.07.
  • bug 734996 - source: get a nonce earlier
in production 2012-03-13 17:00 PDT
  • bug 735201 - Remove leading ../ from symbols path for tegras
  • bug 735421 - Disable Aurora updates until the Aurora 13 has stabilized
in production 2012-03-12 16:00 PDT
  • bug 734417 - enable mobile builds on the profiling branch
  • bug 731617 - No nightly builds on maple branch since 27 Feb
  • bug 732285 - Set MINIDUMP_STACKWALK for Android
  • bug 733668 - Include "ERROR: We tried to download the talos.json file but something failed" and "ERROR 500: Internal Server Error" for Talos hgweb operations to RETRY
  • bug 630518 - l10n verify, update verify, and final verification builders need to set "branch" when reporting to clobberer
in production 2012-03-08 09:00 PT
  • bug 731814 - Add checks that we're not exceeding max # of builders per slave.
  • bug 731617 - Remove win64 for now in maple.
  • bug 731339 - Remove slaves that are not production
  • bug 732730 - Remove non-functional and unwanted pgo_platforms overrides
  • bug 732110 - remove buildbot-configs/mozilla2/mobile
  • bug 728271 - Post to graphs.m.o instead of graphs-old.m.o
  • bug 729144 - Post to graphs.allizom.org.
  • bug 723667 - Add robocop disabled.
  • bug 730050 - TryBuildFactory looks in the wrong place for malloc.log
  • bug 712538 - leaktest parity on try
  • bug 723667 - Use talos.zip for tegras and prep work for talos robocop
in production 2012-03-06 06:30 PT
  • bug 732500 - Enable nightly updates on maple
  • bug 732699 - ESR release automation should push to mirrors automatically
  • bug 730918 - Android on esr10 is busted, no doubt by branding since that always seems to be the problem
  • bug 561754 - Don't download symbols for test runs, pass symbol zip URL as symbols path
  • bug 732516 - l10n verification shouldn't rsync zip files
  • bug 732468 - Add the ridiculous "abort: error:" to the list of hg errors that trigger RETRY
in production 2012-03-01 7:30 PT
  • bug 721360 - Do what changeset 9a0c428bdb69 really wanted to do.
  • bug 561754 - Disable symbol download on demand for mozilla-1.9.2 branch.
  • bug 660480 - mark as RETRY for common tegra errors
  • bug 729918 - start_uptake_monitoring builder uses wrong script_repo_revision property.
  • bug 561754 - Download symbols on demand by default for desktop unittests.
in production 2012-02-27 7:45 PT
  • bug 729077 - recycle talos-r4-lion-083 and talos-r3-snow-081 as mac-signing[12]
  • Fix up staging and preproduction test slave lists.
  • bug 729426 - Do periodic PGO on services-central
  • bug 727580 - linux-android for esr10, without merging 11.0 to m-r.
in production 2012-02-21 9:30 PT
  • bug 719511 - add optional reboot command to ScriptFactory
  • bug 725292 - some repacks failed in 11.0b2 because of missing tokens
  • bug 728104 - AggregatingScheduler resets its state on reconfig
  • bug 722608 - Remove android signature verification
  • bug 719260 - Investigate why updates builder triggered twice for 10.0b5
  • bug 719511 - Reenable peptest + add reboot_command
  • bug 712678 - android-xul different update channel from android
in production 20120217 1148 PST
  • bug 721822 - remove talos_from_code.py from the tools repo
in production 20120214 1245 PST
  • bug 726901 - adjust resolution for reftests to 1600x1200
  • bug 689989 - restore /system/etc/hosts on testing tegras
in production 20120213 1200 PST
in production 20120208 01:20 PST
in production 20120207 13:00 PST
  • bug 719443 - add robocop unitttest testtype
  • bug 715715 - download & install robocop for robocop test suites
  • bug 725046 - Re-enable mobile aurora updates
  • bug 554324 - Only set MOZ_PKG_VERSION when appVersion != version
  • [BACKED OUT] - bug 719511 - optional ScriptFactory reboot().
in production 20120202 15:50 PST
  • bug 723743 - android native to en-US (no multilocale); disable android-xul single locale repacks.
  • bug 719697 - --disable-tests on android* l10n-mozconfigs.
  • bug 723277 - don't enable remot

Android Testing

Web Server Cluster

Revision Bug # When Who
3efbac1f685a unknown unknown unknown

Update Procedure:

ssh to bm-remote-talos-webhost-01
cd /var/www/html/talos
hg pull && hg up
rsync -azf --delete . bm-remote-talos-webhost-02:/var/www/html/.
rsync -azf --delete . bm-remote-talos-webhost-02:/var/www/html/.

Servers:

  • bm-remote-talos-webhost-01.build.mozilla.org
  • bm-remote-talos-webhost-02.build.mozilla.org
  • bm-remote-talos-webhost-03.build.mozilla.org

clientproxy servers

Production

Talos Rev Pageloader Rev Taras Bench Rev sut_tools Bug # When Who
2a995b4ed124 31249cbe4f19 bfc910cd8dd3 ae5d6911905a talos: bug 629503 20110202 23:00 PDT bear

Pending

Talos Rev Pageloader Rev Taras Bench Rev sut_tools Bug # When Who

Servers:

  • bm-foopy01.build.mozilla.org
  • bm-foopy02.build.mozilla.org
/builds/cp
/builds/talos-data/talos
/builds/talos-data/talos/pageloader@mozilla.org
/builds/talos-data/talos/bench@taras.glek
/builds/sut_tools