Auto-tools/Projects/MozmillBuildbot: Difference between revisions

update to reflect several areas of recent work
(→‎Long-term Recommendations: note non-actionability of long-term recommendations)
(update to reflect several areas of recent work)
Line 17: Line 17:
== Outstanding Issues ==
== Outstanding Issues ==


* Mozmill build fails on download:
None at the moment ;)
Traceback (most recent call last):
  File "/home/jhammel/mozilla/src/buildbot/buildbot/buildbot/process/buildstep.py", line 689, in startStep
    d.addCallback(self._startStep_2)
  File "/home/jhammel/mozilla/src/buildbot/lib/python2.6/site-packages/Twisted-9.0.0-py2.6-linux-i686.egg/twisted/internet/defer.py", line 190, in addCallback
    callbackKeywords=kw)
  File "/home/jhammel/mozilla/src/buildbot/lib/python2.6/site-packages/Twisted-9.0.0-py2.6-linux-i686.egg/twisted/internet/defer.py", line 181, in addCallbacks
    self._runCallbacks()
  File "/home/jhammel/mozilla/src/buildbot/lib/python2.6/site-packages/Twisted-9.0.0-py2.6-linux-i686.egg/twisted/internet/defer.py", line 323, in _runCallbacks
    self.result = callback(self.result, *args, **kw)
--- <exception caught here> ---
  File "/home/jhammel/mozilla/src/buildbot/buildbot/buildbot/process/buildstep.py", line 712, in _startStep_2
    skip = self.start()
  File "/home/jhammel/mozilla/src/buildbot/buildbotcustom/buildbotcustom/steps/misc.py", line 385, in start
    url = self.url_fn(self.build)
  File "/home/jhammel/mozilla/src/buildbot/buildbotcustom/buildbotcustom/process/factory.py", line 6623, in get_fileURL
    fileURL = build.source.changes[-1].files[0]
exceptions.IndexError: tuple index out of range


The patch at http://k0s.org/mozilla/hg/buildbotcustom/.hg/patches/rev/71b8c045d03a reveals <tt>build.source.changes</tt> is in fact an empty tuple.  This is from ''force build'' through the web.
== How Buildbot Works ==


This may be worked around by using a static URL for the tarball when there is not a change source.  See http://k0s.org/mozilla/hg/buildbotcustom/.hg/patches/rev/1eea0e4dc4cd for a quick patch.  Ideally, the patch would construct the URL needed dynamically, but its a WIP.
''notes from integrating the MozmillFactory [STUB]''


However, this patch causes another problem as the crashreporter symbols cannot be found:
wget --progress=dot:mega -N --no-check-certificate http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-central/firefox-3.7a4pre.en-US.linux-i686.crashreporter-symbols.zip


== Tips ==
== Tips ==


* buildbotcustom is not a python package.  On the VM, it is installed in <tt>/tools/buildbotcustom/buildbotcustom</tt> which is added on <tt>cltbld</tt> login to <tt>PYTHONPATH</tt>.  On a local machine, I use a [http://pypi.python.org/pypi/virtualenv|virtualenv] to isolate python development.  I download buildbotcustom to <tt>$VIRTUAL_ENV/buildbotcustom/buildbotcustom</tt> and then add <tt>$VIRTUAL_ENV/buildbotcustom</tt> to the <tt>PYTHONPATH</tt> in the <tt>$VIRTUAL_ENV/bin/activate</tt> file so that the code is available within the virtualenv.
* buildbotcustom is not a python package.  On the VM, it is installed in <tt>/tools/buildbotcustom/buildbotcustom</tt> which is added on <tt>cltbld</tt> login to <tt>PYTHONPATH</tt>.  On a local machine, I use a [http://pypi.python.org/pypi/virtualenv|virtualenv] to isolate python development.  I download buildbotcustom to <tt>$VIRTUAL_ENV/buildbotcustom/buildbotcustom</tt> and then add <tt>$VIRTUAL_ENV/buildbotcustom</tt> to the <tt>PYTHONPATH</tt> in the <tt>$VIRTUAL_ENV/bin/activate</tt> file so that the code is available within the virtualenv.
* forcing builds through the web does not work.  This is due to the fact that the build and the crashsymbols are downloaded must be downloaded from the [http://stage.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/ tinderbox builds] so the location of the desired build is unknown as they are indexed by time stamps (seconds since epoch).  <tt>buildbot sendchange</tt> should instead be used to trigger a build for testing purposes.


== Outstanding Work ==
== Outstanding Work ==


* Add corrections to https://wiki.mozilla.org/Buildbot and linked pages
* Add corrections to https://wiki.mozilla.org/Buildbot and linked pages.  First this page should be finished (presumedly near project completion) and then the both pages audited to figure out what needs to be done.




Line 74: Line 56:
         ))
         ))
However, for the case of TTW force builds, <tt>build.source.changes</tt> is an empty tuple, so the step fails with an exception.  In general, if <tt>build.source.changes</tt> is empty, a URL should be constructed from the platform and other variables appropriate to the build
However, for the case of TTW force builds, <tt>build.source.changes</tt> is an empty tuple, so the step fails with an exception.  In general, if <tt>build.source.changes</tt> is empty, a URL should be constructed from the platform and other variables appropriate to the build
* '''Make a paster template for buildbot testing:''' As a generalization, the production [http://hg.mozilla.org/build/buildbot-configs/ buildbot configs] are less appropriate for testing and developing Mozilla's buildbot.  A template could be created to make it easier to make testing master and slave instances.  Files should include <tt>master.cfg</tt> and a <tt>forcebuild</tt> script for the buildslave.  The template additionally serves as instructions for future generations.
947

edits