I don't understand popen2 :(

[Donn Cave]

Quoth marco at atmosp.physics.utoronto.ca:
| In article <t8m0p6j29c9ia3 at corp.supernews.com>, Penfold <spam at spam.com>
| writes:
| >This is simply the usual nightmare of dealing with pipes between processes.
| [snip]
|
| Thanks for the explanation, I suspected something along those lines...
|
|> Solution ... I tend to alter the pipes to no longer be blocking ... I'm not
|> sure what other people do in these cases.  You
|> can use fctl to do this but I forget the flags you want, something like
|> NDELAY and O_NOBLOCK ... I have none of
|> my sample code to hand.  Look at man fcntl.
|
| Good tip! I've started looking into this, but if anyone has some
| examples it would be a great help (I would assume people run into
| these problems all the time? Unfortunately my Python books -- the
| O'Reilly ones -- make nary a mention of fcntl.)

Non-blocking may not help at all.  I personally prefer to always
leave things in a blocking state and use select();  that's essentially
similar to non-blocking, though a little better documented, so it's
just as likely to be of no use.  Both of these techniques apply to
UNIX file descriptors (see file object fileno()), and if you're going
that way I would also use the UNIX I/O functions - posix.read(),
write() etc. and stay away from the file object.  The UNIX read()
will incidentally return what's in the pipe, instead of blocking
for more input as the file object read() does.


|> Alternatively, just read the pipes in a sensible order :-)
|
| Well, I tried to do that, and I think I got it working.
| However, it doesn't seem to be completely reliable, i.e.
| I have to run popen2.popen3 many times and once in a while
| it will get stuck, even if the order is right 99% of the time :(

Some applications of this just won't work.  The most common problem
is when you want to leave both the input and output units open, and
the process on the other end of the pipe mysteriously fails to write
output you expected.  That's because its output is block buffered,
and it won't be actually written to the pipe until the buffer is
full or the application exits.  You can't control that from the
other end of the pipe.  If you can close the other process' input
pipe, then it may exit and flush its buffers.  The other problem,
and this one select() can help, is when you're blocking on one
pipe and the other process is blocking on the other, either waiting
to write more data in a full pipe or waiting to read.

	Donn Cave, donn at oz.net