Reading output from a child process non-blockingly

[Adriaan Renting]

I use the pty module, in combination with select.select.
Search for Pexpect for an elaborate example.
It basically goes something like this:


-------------------------------------------------
import os, select, pty

pid, fd = pty.fork()
fd_eof = 0
if pid == 0:
 os.execvp('ls',['ls']) # child process
else: # parent proces
 while not fd_eof:
   ready = select.select([fd], [], [], 0.25)
   if fd in ready[0]:
     text = os.read(fd, 1024)
     if text == '':
       fd_eof = 1
     else: print text
-------------------------------------------------------------
In 2.3.4 this exits with an exception OSError: [Errno 5] Input/output
error after showing the 'ls' output, I have a problem that this works in
2.3.4 but in 2.3.5 this just keeps running indefinately. In my own code
I handle the OSError by setting fd_eof=1. This onyl works on Unix.


Adriaan Renting        | Email: renting at astron.nl
ASTRON                 | Phone: +31 521 595 217
P.O. Box 2             | GSM:   +31 6 24 25 17 28
NL-7990 AA Dwingeloo   | FAX:   +31 521 597 332
The Netherlands        | Web: http://www.astron.nl/~renting/
>>> Yuan HOng <hongyuan1306 at gmail.com> 06/29/05 10:08 AM >>>
In my program I have to call an external program and parse its output.
For that I use the os.popen2 function, and then read the output
stream.

But the complexity is that the external program gives back its output
in a piecemeal manner, with long delays between the outputs. In the
main program I want to therefore read the output in a non-blocking
manner, to read as many bytes as the child process is spitting out.

The question is, how can I achieve that?

I tried use select.select on the output stream returned by os.popen2,
but it returns a readable file descriptor only after the whole child
process ends.

Here is a script simulating the external program:

test.py:
import sys, time
print 'hello\n'*500
sys.stdout.flush()
time.sleep(100)
print 'world\n'*500 

And here is what I am tring to do in the main program to read its
output:

import os, select
cmd = 'python test.py'
pin, pout = os.popen2(cmd)
while not select.select([pout], [], [], some_timeout)[0]:
  pass
pout.readline()

I hope to get the first return very soon, before the external program
sleeps, but instead only after the whole program exits do I get any
output.

Can anyone give me a hint?

-- 
Hong Yuan

         
www.homemaster.cn
-- 
http://mail.python.org/mailman/listinfo/python-list