[Python-Dev] Stable buildbots

[David Bolen]

Nick Coghlan <ncoghlan at gmail.com> writes:

> Do we have any idea why the workaround to avoid the popup windows
> stopped working? (assuming it ever worked reliably - I thought it did,
> but that impression may have been incorrect)

Oh, the pop-up handling for the RTL dialogs still seems to be working
fine (at least I haven't seen any since I put it in place).  That, plus
the original buildbot tweaks to block any OS popups still looks solid
for avoiding any dialogs that block a test process.

This is a completely separate issue, though probably around just as
long, and like the popup problem its frequency changes over time.  By
"hung" here I'm referring to cases where something must go wrong with
a test and/or its cleanup such that a python_d process remains
running, usually several of them at the same time.  So I end up with a
bunch of python_d processes in the background (but not with any
dialogs pending), which eventually cause errors during attempts the
next time the same builder is used since the file remains in use.

I expect some of this may be the lack of a good process group cleanup
under Windows, though the root cause may not be unique to Windows.  I
see something very similar reasonable frequency on my OSX Tiger
buildbot as well.  But since the filesystem there can let the build
tree get cleaned and rebuilt even with a stranded executable, the
impact is minimal on subsequent tests than on Windows, though the OSX
processes do burn a ton of CPU.  I run a script on OSX to kill them
off, but that was quick to whip up since in those cases the stranded
processes all end up getting owned by init so it's a simple ps grep
and kill.  In the Windows case I'll probably just set a time limit so
if the processes have been around more than a few hours I figure
they're safe to kill.

-- David