Reading Live Output from a Subprocess

[John O'Hagan]

On Fri, 6 Apr 2012 12:21:51 -0700 (PDT)
Dubslow <bunslow at gmail.com> wrote:

> On Friday, April 6, 2012 3:37:10 AM UTC-5, Nobody wrote:
> 
> > In all probability, this is because the child process (pypy) is
> > buffering its stdout, meaning that the data doesn't get passed to the OS
> > until either the buffer is full or the process terminates. If it doesn't
> > get passed to the OS, then the OS can't pass it on to whatever is on the
> > read end of the pipe. In that situation, there is nothing that the parent
> > process can do about it.
> >
> It's just a short test script written in python, so I have no idea how to
> even control the buffering (and even if I did, I still can't modify the
> subprocess I need to use in my script). What confuses me then is why Perl is
> able to get around this just fine without faking a terminal or similar stuff.
> (And also, this needs to work in Windows as well.) For the record, here's the
> test script:
> ######################################
> #!/usr/bin/python
> 
> import time, sys
> try:
> 	total = int(sys.argv[1])
> except IndexError:
> 	total = 10
> 
> for i in range(total):
> 	print('This is iteration', i)
> 	time.sleep(1)
> 
> print('Done. Exiting!')
> sys.exit(0)
> ######################################
> 

I agree that this kind of thing (IO) can seem unduly difficult to find out how
to do at first. To get your test script working, you just need to flush stdout
from there; add:

sys.stdout.flush()

to your for loop above. 

However, if there is a real process you need to use which you "can't modify",
then it seems you'll have to use one of the more complex solutions offered here.

Regards,

John