[Chicago] capturing output from subprocesses
Noel Thomas Taylor
nttaylor at uchicago.edu
Wed Nov 23 00:49:10 CET 2005
I had not properly understood the difference before in how EOF is
registered when reading from a pipe vs. when reading from a pty. All is
much clearer now and I've implemented the code as you outlined below,
including separate tracking of the stdOut and stdErr buffers.
By the way, I removed one level of nesting by replacing:
for f in rl:
if f == outpi:
o = read(f, 100)
if outpi in rl:
o = read(outpi, 100)
So far there seem to be no ill-effects, but I've been burned making
seemingly innocuous changes before, so I thought I'd bring it up.
Everything is still running fine on all platforms, and my understanding of
these functions is at an all-time high. Still, I'd like to ask you about
one of the more subtle tricks with pseudo-terminals.
I've read that sometimes when you're passing data through a
pseudo-terminal, it will change "\n" to "\r\n" and in this way your data,
especially binary data, can be corrupted.
Since I'm always passing text in my implementation, this isn't a big
problem, and I just replace all "\r" instances with the empty string.
o = read(f, 100).replace("\r","")
But if I *did* want to pass binary data, I'd be in trouble right? The
situation is made more complicated by the fact that the data might be
coming from a remote host.
Do you know a smooth way around this problem?
with thanks and best wishes for the holiday,
> So based on what was said above, this seems to be the better implementation:
> # finished flag
> end = 0
> for f in rl:
> if f == outpi:
> # we might not be able to read the pty
> # so we trap the error and signal end, we
> # don't break because there might be something
> # able to be read from "error" stream
> o = read(f, 100)
> except OSError, e:
> end = 1
> output += o
> elif f == errpi:
> # error stream is a pipe so we get a proper EOF
> # and can signal end from that
> e = read(f, 100)
> if e: error += e
> else: end = 1
> # exit the 'while 1' when nothing left to be read
> if end:
> Also, it's possible that if you have a very large amount of amount in
> one of the buffers that the other one will signal 'end' before you
> finish reading. So if you want it to be perfect, you should track
> 'end' for each (stdout and stderr) buffer individually and adjust the
> read's and read fdset accordingly.
> Chicago mailing list
> Chicago at python.org
More information about the Chicago