Python 2.2.1 and select()
bockman at virgilio.it
Tue Mar 25 10:38:47 CET 2008
Il Mon, 24 Mar 2008 17:58:42 -0400, Derek Martin ha scritto:
> Hi kids!
> I've got some code that uses select.select() to capture all the output
> of a subprocess (both stdout and stderr, see below). This code works as
> expected on a variety of Fedora systems running Python > 2.4.0, but on a
> Debian Sarge system running Python 2.2.1 it's a no-go. I'm thinking
> this is a bug in that particular version of Python, but I'd like to have
> confirmation if anyone can provide it.
> The behavior I see is this: the call to select() returns: [<file
> corresponding to sub-proc's STDOUT>]  
> If and only if the total amount of output is greater than the specified
> buffer size, then reading on this file hangs indefinitely. For what it's
> worth, the program whose output I need to capture with this generates
> about 17k of output to STDERR, and about 1k of output to STDOUT, at
> essentially random intervals. But I also ran it with a test shell
> script that generates roughly the same amount of output to each file
> object, alternating between STDOUT and STDERR, with the same results.
> Yes, I'm aware that this version of Python is quite old, but I don't
> have a great deal of control over that (though if this is indeed a
> python bug, as opposed to a problem with my implementation, it might
> provide some leverage to get it upgraded)... Thanks in advance for any
> help you can provide. The code in question (quite short) follows:
> def capture(cmd):
> buffsize = 8192
> inlist = 
> inbuf = ""
> errbuf = ""
> io = popen2.Popen3(cmd, True, buffsize) inlist.append(io.fromchild)
> while True:
> ins, outs, excepts = select.select(inlist, , ) for i in ins:
> x = i.read()
> if not x:
> if i == io.fromchild:
> inbuf += x
> if i == io.childerr:
> errbuf += x
> if not inlist:
> if io.wait():
> raise FailedExitStatus, errbuf
> return (inbuf, errbuf)
> If anyone would like, I could also provide a shell script and a main
> program one could use to test this function...
>From yor description, it would seem that two events occurs:
- there are actual data to read, but in amount less than bufsize.
- the subsequent read waits (for wathever reason) until a full buffer can
be read, and therefore lock your program.
Try specifying bufsize=1 or doing read(1). If my guess is correct, you
should not see the problem. I'm not sure that either is a good solution
for you, since both have performance issues.
Anyway, I doubt that the python library does more than wrapping the
system call, so if there is a bug it is probably in the software layers
More information about the Python-list