[Python-Dev] Possible language summit topic: buildbots
"Martin v. Löwis"
martin at v.loewis.de
Sun Oct 25 10:47:10 CET 2009
Mark Dickinson wrote:
> Would it be worth spending some time discussing the buildbot situation
> at the PyCon 2010 language summit? In the past, I've found the
> buildbots to be an incredibly valuable resource; especially when
> working with aspects of Python or C that tend to vary significantly
> from platform to platform (for me, this usually means floating-point,
> and platform math libraries, but there are surely many other things it
> applies to). But more recently there seem to have been some
> difficulties keeping a reasonable number of buildbots up and running.
> A secondary problem is that it can be awkward to debug some of the
> more obscure test failures on buildbots without having direct access
> to the machine. From conversations on IRC, I don't think I'm alone in
> wanting to find ways to make the buildbots more useful.
These are actually two issues:
a) where do we get buildbot hardware and operators?
b) how can we reasonably debug problems occurring on buildbots
For a), I think we can solve this only by redundancy, i.e. create
more build slaves, hoping that a sufficient number would be up at
any point in time.
So: what specific kinds of buildbots do you think are currently lacking?
A call for volunteers will likely be answered quickly.
> So the question is: how best to invest time and possibly money to
> improve the buildbot situation (and as a result, I hope, improve the
> quality of Python)?
I don't think money will really help (I'm skeptical in general that
money helps in open source projects). As for time: "buildbot scales",
meaning that the buildbot slave admins will all share the load, being
responsible only for their own slaves.
On the master side: would you be interested in tracking slave admins?
> What could be done to make maintenance of build
> slaves easier?
This is something that only the slave admins can answer. I don't think
it's difficult - it's just that people are really unlikely to contribute
to the same thing over a period of five years at a steady rate. So we
need to make sure to find replacements when people drop out.
> Or to encourage interested third parties to donate
> hardware and time?
Again: I think a call for volunteers would do (Steve, if you are
reading this, please hold back just a few days before actually
making such a call :-)
> Are there good alternatives to Buildbot that might
> make a difference?
I think people have started working on such a thing. There are
certainly alternatives; I'm fairly skeptical that they are *good*
alternatives (but then, I'm the one who set up the buildbot
installation in the first place).
> What do other projects do?
I think that's really difficult to compare, since their testing
often has a very different scope. I think CruiseControl is widely
> These are probably the wrong questions; I'm hoping that a discussion
> would help produce the right questions, and possibly some answers.
I think these are good questions - just not for the summit. Setting
up such a system is, conceptually, easy. It's also just a little work
to set it up initially; the difficult part then is to keep it running
(and no, a system where anybody can just post test results at any time
without prior registration is *still* difficult to keep running).
The source of the problem is that such a system can degrade without
anybody taking action. If the web server's hard disk breaks down,
people panic and look for a solution quickly. If the source control
is down, somebody *will* "volunteer" to fix it. If the automated
build system produces results less useful, people will worry, but
not take action.
More information about the Python-Dev