popenX() misunderstanding on win32

[Steve Holden]

"Jon Nicoll" <jkn at nicorp.f9.co.uk> wrote ...
> Hello there
>
>     I'm trying to get a small python script running under WinNT and am
> having trouble due to my lack of understanding of the mechanism of
> popen2() etc. I know that previous versions of popenX() have been
> broken but I don't think that's the problem.
>
> In its simplest form:
>
> Say I have a console program UPPER.EXE, which loops round echoing
> standard input to standard output:
>
> c:\> upper
> qwertyuiop    # my typing
> QWERTYUIOP    # the program
> etaions       # me
> ETAOINS       # the program
> ...
> ^C
> C:\>
>
> Then, how can I use popenX() to invoke the program, and write and read
> to 'stdin' and 'stdout', to duplicate the above? my experiments so
> far, involving os.popenX() and win32pipe.popenX(), haven't got me
> anywhere - I either get nothing when I do a read, or the process hangs
> waiting for input that I think should alread be there.
>

Jon:

The synchronisation of input and output via pipes from a single process can
be very tricky to solve if you don't know what to expect from the program.
Even when you do, you will find that the OS's buffering system can get in
your way. This is a classic problem in half-duplex protocol design.

The problem is that when you run the "driven" program (in this case,
upper.exe) on a terminal you generally get non-buffered or line-buffered
I/O, so everything works fine. You can clearly see when more input is
required, because you see the output corresponding to your prior input. Once
pipes are involved, your "driven" process has to see the input before it can
respond, but when your Python program writes to the pipe there's no way of
telling the I/O system to flush, except by explicit use of flush(). This can
take care of the output side, but if the program reading your input from a
pipe doesn't know it's interactive it will not be likely to be doing the
same sort of thing.

Thus your program can get blocked, waiting for output to arrive from the
driven program, when the output is just sitting in an I/O buffer waiting to
be flushed by more output. Classic deadlock, since more output will only be
produced when the driven program will only produce when it sees more input,
which your program will only produce when it sees the output it's waiting
for.

Thus what you are looking to do is, in the general case, tricky. If you can
force the use of unbuffered I/O in both directions and you know pretty much
what to expect or when "turnaround" is required, you can usually get things
working. Otherwise you are left resorting to the use os pseudo-teletypes on
Unix-style platforms, and pretty much buggered under Windows.

regards
 Steve
--
-----------------------------------------------------------------------
Steve Holden                                 http://www.holdenweb.com/
Python Web Programming                http://pydish.holdenweb.com/pwp/
-----------------------------------------------------------------------