[Python-Dev] please take a look at buildbot result

[David Bolen]

Bill Janssen <janssen at parc.com> writes:

>Ronald Oussoren <ronaldoussoren at mac.com> wrote:

>> As Antoine noted the test failures are unexpected, could you check if
>> the tests pass if you do the build and testrun manually?

> What about readline?  Darwin doesn't have it (or rather, it has a
> different one).  Why is that 'skip' unexpected?

For what it's worth, for the libraries, I used the build-installer
script to build the same versions of libraries that it uses for the
binary installer, and installed them in /usr/local for my buildbot to
use, until there's enough time to have the buildbot build such local
libraries within its own tree.  That at least matches up the external
dependencies for buildbot builds with that used by the eventual binary
installer.

For my buildbot, generally the only unexpected skips are for
test_bsddb185 (which feels right to me - I don't have that version of
the library, nor do I think has the binary installer) and test_ioctl
(which I have no idea on yet).

>> IMHO it would be better to do a framework build for at least some of
>> the OSX buildbots because that's what is in the binary installer for
>> OSX.
>
> Yes, that sounds good to me, too.  But how do we make that a standard
> test so that appropriately enabled buildbot slaves will do it
> automatically?  Something that gets configured in the build master?

This came up in the earlier discussion, and while I do still think
that's a better long term approach, I suspect that with respect to
test coverage and code generation the non-framework build for the
interpreter is a fair representation for testing.

I suspect much of this is just a builder change on the master (to use
the right Makefile targets for generating the framework build), but
just given experience to date getting the binary installer building
under buildbot, there may be some unexpected environmental stuff.

Ideally we'd work up a way to do a universal framework build (though
I'm not sure what if any extra support may be needed to have the
system use the framework if not installed in a standard location), and
then for full testing, maybe even use the makefile target that tests
both the Intel and PPC path (the latter via Rosetta on an Intel
system).

What I'm not sure about at the moment is how much different in terms
of testing such a setup might be (if, for example, any extra work is
needed to be able to test a framework build while still local in the
buildbot tree and not in a normal framework location), so how much
energy is worth putting into it, especially if that might use resource
among those able to resolve some of the open code issues.

For my part, if there's free cycles I'd personally rather address the
external library issue first, as opposed to the framework build stuff,
since that feels more fragile to me (in terms of the buildbot
environment for the Mac) at the moment.  Over the past few weeks I'm
also fairly sure that I'm finding stranded python test processes (not
too dissimilar as on the Windows buildbots) that are bogging down the
buildbot and thus more detrimental to ongoing buildbot operation than
the lack of framework builds, so there may be other unknown issues more
valuable to hit as we get more experience.

-- David