Reading output from a child process non-blockingly

[Yuan HOng]

In my program I have to call an external program and parse its output.
For that I use the os.popen2 function, and then read the output
stream.

But the complexity is that the external program gives back its output
in a piecemeal manner, with long delays between the outputs. In the
main program I want to therefore read the output in a non-blocking
manner, to read as many bytes as the child process is spitting out.

The question is, how can I achieve that?

I tried use select.select on the output stream returned by os.popen2,
but it returns a readable file descriptor only after the whole child
process ends.

Here is a script simulating the external program:

test.py:
import sys, time
print 'hello\n'*500
sys.stdout.flush()
time.sleep(100)
print 'world\n'*500 

And here is what I am tring to do in the main program to read its output:

import os, select
cmd = 'python test.py'
pin, pout = os.popen2(cmd)
while not select.select([pout], [], [], some_timeout)[0]:
  pass
pout.readline()

I hope to get the first return very soon, before the external program
sleeps, but instead only after the whole program exits do I get any
output.

Can anyone give me a hint?

-- 
Hong Yuan

大管家网上建材超市
www.homemaster.cn