Reading output from a child process non-blockingly

Adriaan Renting renting at
Thu Jun 30 09:47:33 CEST 2005

I use the pty module, in combination with
Search for Pexpect for an elaborate example.
It basically goes something like this:

import os, select, pty

pid, fd = pty.fork()
fd_eof = 0
if pid == 0:
 os.execvp('ls',['ls']) # child process
else: # parent proces
 while not fd_eof:
   ready =[fd], [], [], 0.25)
   if fd in ready[0]:
     text =, 1024)
     if text == '':
       fd_eof = 1
     else: print text
In 2.3.4 this exits with an exception OSError: [Errno 5] Input/output
error after showing the 'ls' output, I have a problem that this works in
2.3.4 but in 2.3.5 this just keeps running indefinately. In my own code
I handle the OSError by setting fd_eof=1. This onyl works on Unix.

Adriaan Renting        | Email: renting at
ASTRON                 | Phone: +31 521 595 217
P.O. Box 2             | GSM:   +31 6 24 25 17 28
NL-7990 AA Dwingeloo   | FAX:   +31 521 597 332
The Netherlands        | Web:
>>> Yuan HOng <hongyuan1306 at> 06/29/05 10:08 AM >>>
In my program I have to call an external program and parse its output.
For that I use the os.popen2 function, and then read the output

But the complexity is that the external program gives back its output
in a piecemeal manner, with long delays between the outputs. In the
main program I want to therefore read the output in a non-blocking
manner, to read as many bytes as the child process is spitting out.

The question is, how can I achieve that?

I tried use on the output stream returned by os.popen2,
but it returns a readable file descriptor only after the whole child
process ends.

Here is a script simulating the external program:
import sys, time
print 'hello\n'*500
print 'world\n'*500 

And here is what I am tring to do in the main program to read its

import os, select
cmd = 'python'
pin, pout = os.popen2(cmd)
while not[pout], [], [], some_timeout)[0]:

I hope to get the first return very soon, before the external program
sleeps, but instead only after the whole program exits do I get any

Can anyone give me a hint?

Hong Yuan


More information about the Python-list mailing list