popen with separete stdout/stderr deadlock.

[Donn Cave]

Quoth Eyal Lotem <eyal at hyperroll.com>:
| When running popen with a separate stdout/stderr, can I use read() on the 
| resulting stdout file? it seems to 'hang', and I'm assuming stderr's buffer 
| is stuck on write (as it is all filled up), and my stdout read() is stuck 
| on waiting for EOF, which will never come.  Deadlock.
|
| Is this really what's happening and how can it be avoided?

Deadlock is real.  I don't know about popen with separate stdout/stderr,
but otherwise the circumstances you describe are conceivable.

- When you use file.read(), you will wait for EOF before anything returns,
  regardless of what else might be going on.
- When you use posix.read(fileno), you will immediately get anything that
  has been flushed to the pipe.
- Nothing will retrieve data that has not been flushed to the pipe,
  whether because the application is blocking on stderr write or simply
  due to normal block buffering.  A pty device in place of the pipe may
  help with the latter, but can have its own problems.
- When the application is writing to two separate pipes, use select(),
  on the file descriptor, to know which one has data.
- When trying to do something like this on Windows, say so, preferably
  in the Subject so we don't have to read about your troubles.

	Donn Cave, donn at drizzle.com