Capturing stdout incrementally

[David Bolen]

Josiah Carlson <jcarlson at uci.edu> writes:

> > I am on Windows by the way, so the utilities are printing to the windows
> > command shell.
> 
> By default, popen in Windows buffers everything coming from the called
> app.  I believe you can use various Windows system calls in pywin32 in
> order to get line buffered output, but I've never done it, so have
> little additional advice other than "check out pywin32".

Typically this behavior on Windows (and at least in my experience on
Unix) comes not from popen buffering its input from the called
program, but the called program buffering its output when it isn't a
normal console.  I'm not sure if this is default behavior from the C
library, but it's particularly true of cross-platform code that often
uses isatty() to check.

You can even see this behavior with Python itself (e.g., if you pipe a
copy of python to run a script).

In general, the only way to fix this is through the program being
called, and not the program doing the calling.  In some cases, the
problem may have a way to disable buffering (for example, the "-u"
command line option with Python), or if it's something you have source
for you can explicitly disable output buffering.  For example, Perl
has no command line option, but you can add code to the script to
disable output buffering.

On Unix, a classic way around code for which you have no source is to
run it under expect or some other pty-simulating code rather than a
simple pipe with popen.  I'm not sure if there's a good pty/expectish
module that works well under Windows (where simulating what a
"console" is can be tougher).

I've only got an evaluation copy of perforce lying around, but I don't
immediately see any way to control its buffering via command line
options.

-- David