[New-bugs-announce] [issue7448] when piping output between subprocesses some fd is left open blocking forever

[Clovis Fabricio]

New submission from Clovis Fabricio <nosklo+python at gmail.com>:

Suppose I want to simulate the following shell pipe using the subprocess
module:

`grep -v not | cut -c 1-10`

The documentation example here
http://docs.python.org/library/subprocess.html#replacing-shell-pipeline
Implies that I want to run this:

grep_process = Popen(["grep", "-v", "not"], stdin=PIPE, stdout=PIPE)
cut_process = Popen(["cut", "-c", "1-10"], stdin=p1.stdout, stdout=PIPE)
grep_process.stdin.write('Hello World\n')
grep_process.stdin.close()
result = cut_process.stdout.read() # blocks forever here
assert result == "Hello Worl\n"

When grep_prcoess starts, two file descriptors are created, one 
for each end of the pipe. Lets call those `grep-stdin-w` and
`grep-stdout-r`.
When I run cut_process, `grep-stdout-r` gets passed as cut_process sdtin. 
Since `close_fds=False` by default, The effect of that is that
cut_process also 
inherits `grep-stdin-w`. So `grep` can't die even if I explicty run
`grep_process.stdin.close()`
because `grep-stdin-w` stdin is still inside cut_process (`cut` ignores
the extra open fd).

Passing `close_fds=True` to the second Popen call makes the code work
perfectly.
`close_fds=True` should be the default on unix systems, because it
closes files descriptors that have nothing to do with the process, leaving
stdin, stdout and stderr working, which is the intended behaviour of
most code.

----------
components: Library (Lib)
messages: 96032
nosy: nosklo
severity: normal
status: open
title: when piping output between subprocesses some fd is left open blocking forever
type: behavior
versions: Python 2.7, Python 3.2

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue7448>
_______________________________________