[Python-Dev] Still looking for volunteer to run Windows buildbot

[Tim Peters]

[Tim]
>> Setup is hellish,

[Trent]
> Agreed, though I have everything going with my own testing buildbot
> master (everything for the trunk build that is). My only remaining
> problem is that I can't connect to python.org's master. (Following up
> with Martin.)

Looks like that got fixed.

>> The second-worst part was figuring out which parts of various software
>> docs could be ignored.

> :)
> Did you apply the Berkeley DB patches to your db-4.2.52 sources?

Ah, _which_ patches?  As with my buildbot Wiki page, I write down
everything I do if there's a good chance I may need to do it again. 
So, e.g., these are my words in PCbuild\readme.txt:

    As of 11-Apr-2004, you also need to download and manually apply two
    patches before proceeding (and the sleepycat download page tells you
    about this).  Cygwin patch worked for me.  cd to dist\db-4.2.52 and
    use "patch -p0 < patchfile" once for each downloaded patchfile.

It's possible that there are more patches needed since then, but if so
I wouldn't know about that (last time I built the externals in the
_bsddb part was indeed 11-Apr-2004).

This touches on something we (including Martin) should think about: 
it's very painful to build a full Python on Windows because of these
external packages, each with its own unique and involved compile
dance, and I expect it's a _huge_ barrier for getting Windows buildbot
volunteers.  You have to do real, messy, tedious work to get beyond
this part.

Something we might be able to do instead is have just one person per
external project endure the pain of building it, and then have them
check in the whole post-compilation post-test project directory tree. 
Everyone else (presumably including Windows buildbot slaves) using the
same compiler could then just check out the result.

>> I recorded all that remained here:
>>
>>     http://wiki.python.org/moin/BuilbotOnWindows

> correction for others:
>     http://wiki.python.org/moin/BuildbotOnWindows

The URL I gave was correct at the time I sent it ;-)  You're right
that I renamed the page later, and good eye!

> I'm worried about the load this is going to cause on this machine with a
> new build for every checkin (granted they are serialized so not for
> *every* checkin). The full (doubled) test suite takes a *long* time to
> run on Windows. It looks like it took about 25 minutes on your box. It
> is taking over 40 minutes on one of my machines here. :( This Windows
> box needs an enema.

Its current second test run is pretty pointless.  Would be fine by me
if we changed test.bat from

call rt.bat -d -uall -rw

to

call rt.bat -d -q -uall -rw

"-q" would cause it to run the tests only once.

What's worse is when checkins cause both trunk and branch rebuilds. 
That can bring my zippy box to its knees, particularly when two of the
disk-intensive tests (like test_largefile) run at the same time.

> Remaining TODOs:
> - get connection to python.org master working
> - make sure the python24-maint branch one works

Works for me, although (as the Wiki page says) you also need to copy
zlib-1.2.3 into the branch build area.

> - see about the load issue
> - get the build slaves running as a Windows service

Mark Hammond has a patch to the buildbot project toward this end:

<http://sf.net/tracker/index.php?func=detail&aid=1401121&group_id=73177&atid=537003>

> - update PCbuild/readme.txt

Anything in particular need changing there?