Buildbot/Talos/AddonBuildbotConfiguration

Buildbot

[Addon On Demand Performance Testing Buildbot Waterfall]

[AddonTester Tinderbox Waterfall]

Network Flow

• test request is generated by AMO and pushed into the system via the trigger.cgi script
• addon buildbot master schedules the AMO test request to the appropriate talos slaves
• talos slave picks up test request, tests
• talos slave pushes results to the production graph server
• production graph server packages the results and pushes them directly into the production AMO db
• talos results are reported to the tinderbox waterfall ([AddonTester Waterfall])

Setup Details

Name

talos-addon-master1.amotest.scl1.mozilla.com

Paths

/builds/buildbot/buildbot-configs
/builds/buildbot/production-addons
/builds/buildbot/staging-addons
/var/www/html/trigger/trigger.cgi

Update Production Master

This would be a standard update/reconfig for code changes in buildbotcustom/buildbot-configs.

cd /builds/buildbot/production-addons
cd buildbot-configs
hg pull -u
cd ..
cd buildbotcustom
hg pull -u
cd ..
make reconfig

Generate New Production Master From Scratch

If updating the production master is unsuccessful, then you can start from scratch.

cd /builds/buildbot/production-addons
make stop
cd ..
mv production-addons/master/BuildSlaves.py .
rm -rf production-addons
cd buildbot-configs
hg pull -u
make -f Makefile.setup MASTER_NAME=production-addon_master BASEDIR=/builds/buildbot/production-addons PYTHON=/usr/bin/python2.6 VIRTUALENV=/usr/bin/virtualenv-2.6 BUILDBOTCUSTOM_BRANCH=default BUILDBOTCONFIGS_BRANCH=default virtualenv deps install-buildbot master master-makefile
cd ..
mv BuildSlaves.py production-addons/master/.
cd production-addons
make start

Finish by rebooting the addon talos slaves to force them to connect to the new master

Slaves

addon-r3-fed-001.amotest.scl1.mozilla.com
addon-r3-fed-002.amotest.scl1.mozilla.com
addon-r3-snow-001.amotest.scl1.mozilla.com
addon-r3-snow-002.amotest.scl1.mozilla.com
addon-r3-w7-001.amotest.scl1.mozilla.com
addon-r3-w7-002.amotest.scl1.mozilla.com
addon-r3-w7-003.amotest.scl1.mozilla.com

Slave Maintenance

• slaves should reconnect on reboot
• if changes are made to the production talos slaves managed by releng then the changes will have to be manually rolled out to the addon slave, as they are outside the releng vpn

Trigger Script

/var/www/html/trigger/trigger.cgi

Causes individual addon tests to be pushed into the system. Uses the local apache server.

Code

repository: http://hg.mozilla.org/build/tools

tools/buildfarm/utils/trigger.cgi

Fixes

• restart apache
• check file permissions

Manual Changes to Slaves

In order to support the AMO API, we need to support oauth2, httplib2, simplejson. This is done by doing:

wget http://pypi.python.org/packages/2.4/s/setuptools/setuptools-0.6c11.win32-py2.4.exe#md5=3b1f36f534b475724104dc08008206dd
setuptools-0.6c11.win32-py2.4.exe
easy_install simplejson httplib2 oauth2