[Chicago] capturing output from subprocesses

[Noel Thomas Taylor]

Hi Jess,

I had not properly understood the difference before in how EOF is 
registered when reading from a pipe vs. when reading from a pty. All is 
much clearer now and I've implemented the code as you outlined below, 
including separate tracking of the stdOut and stdErr buffers.

By the way, I removed one level of nesting by replacing:

   for f in rl:
     if f == outpi:
       o = read(f, 100)

with:

   if outpi in rl:
     o = read(outpi, 100)

So far there seem to be no ill-effects, but I've been burned making 
seemingly innocuous changes before, so I thought I'd bring it up.

Everything is still running fine on all platforms, and my understanding of 
these functions is at an all-time high. Still, I'd like to ask you about 
one of the more subtle tricks with pseudo-terminals.

I've read that sometimes when you're passing data through a 
pseudo-terminal, it will change "\n" to "\r\n" and in this way your data, 
especially binary data, can be corrupted.

Since I'm always passing text in my implementation, this isn't a big 
problem, and I just replace all "\r" instances with the empty string.

   o = read(f, 100).replace("\r","")

But if I *did* want to pass binary data, I'd be in trouble right? The 
situation is made more complicated by the fact that the data might be 
coming from a remote host.

Do you know a smooth way around this problem?

with thanks and best wishes for the holiday,

Noel Taylor

> So based on what was said above, this seems to be the better implementation:
>
> # finished flag
> end = 0
>
> for f in rl:
> 	if f == outpi:
> 		# we might not be able to read the pty
> 		# so we trap the error and signal end, we
> 		# don't break because there might be something
> 		# able to be read from "error" stream
> 		try:
> 			o = read(f, 100)
> 		except OSError, e:
> 			end = 1
> 			continue
> 		output += o
> 	elif f == errpi:
> 		# error stream is a pipe so we get a proper EOF
> 		# and can signal end from that
> 		e = read(f, 100)
> 		if e: error += e
> 		else: end = 1
>
> # exit the 'while 1' when nothing left to be read
> if end:
> 	break
>
> ------------------------------------
> Also, it's possible that if you have a very large amount of amount in
> one of the buffers that the other one will signal 'end' before you
> finish reading. So if you want it to be perfect, you should track
> 'end' for each (stdout and stderr) buffer individually and adjust the
> read's and read fdset accordingly.
>
> Jess
> _______________________________________________
> Chicago mailing list
> Chicago at python.org
> http://mail.python.org/mailman/listinfo/chicago
>