Trouble with popen2
Rembrandt Q Einstein
hercules.rockefeller at springfield.??.us
Mon Sep 27 21:27:56 CEST 2004
Rembrandt Q Einstein wrote:
> Rembrandt Q Einstein wrote:
>> I am running an external command and I need to know a) when it is done
>> and b) what it wrote to both stdout and stderr. After a little
>> searching, I found the popen2 module and used the Popen3 class. I'm
>> having trouble with it hanging, though.
>> Here is a very well put (by someone else) posting that describes some
>> I came to a similar conclusion as that poster and his workaround
>> (independently discovered by me) does do the job he requires.
>> However, I need to also read stderr, so I made this sample writer and
>> import sys
>> for line in range(0, int(sys.argv)):
>> print "X" * 120
>> print >>sys.stderr, "hi"
>> import popen2
>> f = popen2.Popen3("./writer.py 50000", True)
>> outs = 
>> errs = 
>> while (f.poll() == -1):
>> errs += f.childerr.readlines()
>> outs += f.fromchild.readlines()
>> This hangs in the childerr.readlines(). Is it blocking? If so, why?
>> In any case, how can I be sure to read ALL data from BOTH stderr and
>> stdout and not be in danger of hanging?
>> PS: I just found that if I swap the order of the readlines()
>> statements, it works. But I don't want to use that until I understand
>> why. I suspect it's a race condition and I don't want to rely on that.
> It's possible that when I have child.readlines() first, it consumes all
> 50000 output lines and then err.readlines() grabs the "hi" and it just
> exists. When I have them the other way, err.readlines() blocks (I
> thought readlines() was non-blocking, though) and it just hangs.
> My real external program isn't nice enough to order the output like
> that, so here's a more realistic writer:
> import sys
> import math
> for i in range(0, int(sys.argv)):
> if math.fmod(i, 100) == 0:
> print >>sys.stderr, "hi"
> print >>sys.stdout, "X" * 120
> The challenge is to write a program that will run this one in a
> subprocess and read all of both stderr and stdout.
Ah--despite my new nick, I am an idiot. I should just use select() on
fromchild and childerr. Something like this:
while f.poll() == -1:
select(, [f.fromchild, f.childerr],, 0)
However, I think I might still miss some data this way.
More information about the Python-list