Auto-tools/Projects/MozmillBuildbot

This page documents the work from https://bugzilla.mozilla.org/show_bug.cgi?id=516984 : Run Mozmill Tests with buildbot

This is an Auto-tools project and a Q2 goal

Accomplished Work

• reworked patch from bug 516984 so that it would cleanly apply: https://bugzilla.mozilla.org/attachment.cgi?id=434790 (March 24, 2010)

• added patch to disable notification: http://k0s.org/mozilla/hg/buildbotcustom/.hg/patches/rev/8ba008fdce67 ; i'm not sure if this is complete or not, but staging instances of buildbot should not email tinderbox or other live recipients

• added patch to allow copying buildbot configuration: http://k0s.org/mozilla/hg/buildbot-configs/rev/aa3ff093fa4b ; useful for making new buildmasters as it clones versus symlinks to a versioned master

• added patch to raise a more verbose exception to diagnose download error: http://k0s.org/mozilla/hg/buildbotcustom/.hg/patches/rev/71b8c045d03a

Outstanding Issues

• Mozmill build fails on download:

Traceback (most recent call last):
 File "/home/jhammel/mozilla/src/buildbot/buildbot/buildbot/process/buildstep.py", line 689, in startStep
   d.addCallback(self._startStep_2)
 File "/home/jhammel/mozilla/src/buildbot/lib/python2.6/site-packages/Twisted-9.0.0-py2.6-linux-i686.egg/twisted/internet/defer.py", line 190, in addCallback
   callbackKeywords=kw)
 File "/home/jhammel/mozilla/src/buildbot/lib/python2.6/site-packages/Twisted-9.0.0-py2.6-linux-i686.egg/twisted/internet/defer.py", line 181, in addCallbacks
   self._runCallbacks()
 File "/home/jhammel/mozilla/src/buildbot/lib/python2.6/site-packages/Twisted-9.0.0-py2.6-linux-i686.egg/twisted/internet/defer.py", line 323, in _runCallbacks
   self.result = callback(self.result, *args, **kw)
--- <exception caught here> ---
 File "/home/jhammel/mozilla/src/buildbot/buildbot/buildbot/process/buildstep.py", line 712, in _startStep_2
   skip = self.start()
 File "/home/jhammel/mozilla/src/buildbot/buildbotcustom/buildbotcustom/steps/misc.py", line 385, in start
   url = self.url_fn(self.build)
 File "/home/jhammel/mozilla/src/buildbot/buildbotcustom/buildbotcustom/process/factory.py", line 6623, in get_fileURL
   fileURL = build.source.changes[-1].files[0]
exceptions.IndexError: tuple index out of range

The patch at http://k0s.org/mozilla/hg/buildbotcustom/.hg/patches/rev/71b8c045d03a reveals build.source.changes is in fact an empty tuple. This is from force build through the web.

This may be worked around by using a static URL for the tarball when there is not a change source. See http://k0s.org/mozilla/hg/buildbotcustom/.hg/patches/rev/1eea0e4dc4cd for a quick patch. Ideally, the patch would construct the URL needed dynamically, but its a WIP.

However, this patch causes another problem as the crashreporter symbols cannot be found:

wget --progress=dot:mega -N --no-check-certificate http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-central/firefox-3.7a4pre.en-US.linux-i686.crashreporter-symbols.zip

Tips

• buildbotcustom is not a python package. On the VM, it is installed in /tools/buildbotcustom/buildbotcustom which is added on cltbld login to PYTHONPATH. On a local machine, I use a [1] to isolate python development. I download buildbotcustom to $VIRTUAL_ENV/buildbotcustom/buildbotcustom and then add $VIRTUAL_ENV/buildbotcustom to the PYTHONPATH in the $VIRTUAL_ENV/bin/activate file so that the code is available within the virtualenv.

Outstanding Work

• Add corrections to https://wiki.mozilla.org/Buildbot and linked pages

Long-term Recommendations

These recommendations are hereby recorded for future use. Their need should be assessed before filed as bugs and made actionable.

• Use templates for buildbot-configs: the configurations in http://hg.mozilla.org/build/buildbot-configs are hard-coded for the setup for VMs. Instead, they could be made templates for easy deployment. Cleanup should also be done so that the configurations are more universal and adding new configurations is easy.

• Make buildbotcustom a python package: reason: there's no reason not to (except legacy reasons, continue reading) and having the python be installable saves the trouble with messing with the PYTHONPATH. There is a considerable downside in practical terms, however; http://hg.mozilla.org/build/buildbotcustom is a top level repository; all files therein would have to be moved to http://hg.mozilla.org/build/buildbotcustom/buildbotcustom with the top level directory containing a setup.py file. While the effort is minimal, it may cause issues for existing installations which, while easily resolvable, might involve some work.

• Allow force build through the web: several factories, including the MozmillFactory, use a buildstep like:

       def get_fileURL(build):
           fileURL = build.source.changes[-1].files[0]
           build.setProperty('fileURL', fileURL, 'DownloadFile')
           return fileURL
       self.addStep(DownloadFile(
        url_fn=get_fileURL,
        filename_property='build_filename',
        haltOnFailure=True,
        name="download_build",
       ))

However, for the case of TTW force builds, build.source.changes is an empty tuple, so the step fails with an exception. In general, if build.source.changes is empty, a URL should be constructed from the platform and other variables appropriate to the build