subprocess.Popen does not close pipe in an error case

[Nobody]

On Tue, 05 Jan 2010 15:50:39 -0800, Steven K. Wong wrote:

> Below, I have a Python script that launches 2 child programs, prog1
> and prog2, with prog1's stdout connected to prog2's stdin via a pipe.
> (It's like executing "prog1 | prog2" in the shell.)
> 
> If both child programs exit with 0, then the script runs to
> completion. But if prog2 exits with non-0, prog1 does not exit and the
> script hangs (i.e. prog1.poll() always returns None) -- unless I
> uncomment the 2 lines marked by XXX to close prog1.stdout.
> 
> I was expecting that I don't have to explicitly close prog1.stdout,
> whether prog2 succeeds or fails. Is the current behavior a bug in the
> subprocess module or is it expected? Or am I doing something wrong?
> 
> Thanks.
> 
> import subprocess
> import time
> 
> # prog1: a program that writes lots of data to the pipe
> cmd = ['zcat', '--force', 'a_large_file']
> prog1 = subprocess.Popen(cmd, bufsize=-1, stdout=subprocess.PIPE)
> 
> # prog2: a program that fails without reading much data from the pipe
> cmd = ['python', '-c', 'import time; time.sleep(10); asdf']
> prog2 = subprocess.Popen(cmd, bufsize=-1, stdin=prog1.stdout,
>         stdout=open('popen.out', 'w'))

I think that you should close prog1.stdout here. Otherwise, there will
be two readers on the pipe (the calling process and prog2). Even if one of
them dies, there's always the possibility that the caller might eventually
decide to read prog1.stdout itself. If you close it in the caller, when
prog2 terminates there will be no readers, and prog1 will get SIGPIPE (or
write() will fail with EPIPE if SIGPIPE is handled).