[Python-Dev] Community buildbots (was Re: User's complaints)

[Neal Norwitz]

On 7/15/06, "Martin v. Löwis" <martin at v.loewis.de> wrote:
> Terry Reedy wrote:
> > That is the goal, but when I watched the buildbot results last spring, the
> > degree of stability (greenness) appeared to vary.  Is it possible to tag
> > particular versions as a 'green' version, or the 'most recent green
> > version' worth playing with?
>
> Don't get confused by these colors. The tree compiled nearly all of the
> time, even if some tests were failing for an extended period of time on
> some platforms.

Right.  It's very rare that the interpreter or stdlib is broken.
There are 19 buildbots currently.  7 are not green.  5 of those are
offline, most with known (to me and the person that donated the
machines) hardware issues ranging from lack of air conditioning, to a
bad router, to no power.  1 is cygwin which is running an old version.
 I just (an hour or so ago) got an account on a cygwin box to help
diagnose the status and figure out if anything within Python or the
tests are broken.  That leaves 1 unexplained failure on a Windows bot.
 This started happening recently after being down for a while.  I
haven't had time to investigate.

The reason why it was not very green in spring was because that's when
we were getting it set up!  The master looks like it was installed at
the end of 2005/beginning of 2006.  It took several months to get rid
of many testing issues.  Tests that couldn't be run in a particular
order, tests that couldn't be run simultaneously (socket), bad
compilation of sqlite/bsddb modules (not in python), misconfigured
systems, tests verifying something that was system dependent, signed
addresses, etc.

Of all the problems there were, I only remember a single problem in
Python.  (There were probably more, I just remember one.)  That was in
test code (xxmodule or something like that.  There were a bunch of
problems with ctypes and/or sqlite when they got integrated having to
deal with these new platforms.  That may be what you are recalling.
Right before alpha 2 was a particularly bad time.

What we mean by stable is that at any time/any stage of development,
if a change goes in that breaks the tests, it is likely to be fixed or
reverted within hours.  The amount of time the build or tests are
broken on the majority of platforms is quite small.  It took a while
to get to this point due to a bunch of flaky tests, but those have
mostly been fixed.  The only known problems are mentioned on the wiki.

When the buildbots fail, we get mail to python-checkins.
Unfortunately, that's mostly spam at this point.  I hope to fix that
at some point.  I also hope to change the main page to give more info
in less space.

n