[Tutor] Question about subprocess

[eryksun]

On Sat, Jan 11, 2014 at 12:46 AM, Danny Yoo <dyoo at hashcollision.org> wrote:
> There is a warning in the documentation on subprocess that might be
> relevant to your situation:
>
>     Warning:
>     Use communicate() rather than .stdin.write, .stdout.read or
> .stderr.read to avoid deadlocks due to any of the other OS pipe
> buffers filling up and blocking the child process.
>
> Reference: http://docs.python.org/2/library/subprocess.html
>
> It's possible that the process is deadlocking due to this situation.
> Do you run into this issue if you use communicate()?

If more than one standard file is piped (not the case for the OP),
`communicate` avoids deadlocks by using `poll` or `select` on POSIX.
On Windows, it uses the current thread to write to stdin and uses
separate threads to read from stdout and stderr.

If only one standard file is piped, then there's no deadlock. It's
just blocked while waiting for the buffer to fill. `communicate` does
nothing special in this case (at least prior to 3.3):

http://hg.python.org/cpython/file/3a1db0d2747e/Lib/subprocess.py#l767

In 3.3, the new `timeout` option for `communicate` also uses the
select/thread implementation.

stdout FILE stream buffering could be a problem if output is
intermittent and the program doesn't `fflush` the buffer. On Linux the
`stdbuf` program may be able to circumvent this. It injects code (i.e.
libstdbuf.so is added to LD_PRELOAD) that calls `setvbuf` before the
target's `main` runs. This allows you to set the stream to line
buffering mode (unless the program itself calls `setvbuf`). I don't
think a similar utility exists on Windows.

http://linux.die.net/man/1/stdbuf