Non-blocking pipes during subprocess handling

Donn Cave donn at
Tue Jan 9 23:53:37 CET 2007

In article <22q5q25jn7dfti18rv1tshj7os5a0gm27r at>,
 Tom Plunket <tomas at> wrote:

> I'm using subprocess to launch, well, sub-processes, but now I'm
> stumbling due to blocking I/O.
> Is there a way for me to know that there's data on a pipe, and possibly
> how much data is there so I can get it?  Currently I'm doing this:
> 	process = subprocess.Popen(
> 		args,
> 		bufsize=1,
> 		universal_newlines=True,
> 		stdout=subprocess.PIPE,
> 		stderr=subprocess.PIPE)
> 	def ProcessOutput(instream, outstream):
> 		text = instream.readline()
> 		if len(text) > 0:
> 			print >>outstream, text,
> 			return True
> 		else:
> 			return False

I think it would be fair to say that your problem is
not due to blocking I/O, so much as buffered I/O.  Since
you don't appear to need to read one line at a time, you
can detect and read data from the file descriptor without
any buffering.  Don't mix with buffered I/O, as this will
throw the select off.  From memory - better check, since
it has been a while since I wrote anything real like this
(or for that matter much of anything in Python) --

import select
def ProcessOutput(instream, outstream):
    fdr = [instream.fileno()]
    (r, w, e) =, [], [], 0.0)
    for fd in r:
        text =, 4096)

   Donn Cave, donn at

More information about the Python-list mailing list