what happens to Popen()'s parent-side file descriptors?

[Nobody]

On Wed, 13 Oct 2010 14:58:57 -0700, Roger Davis wrote:

> My understanding is that this functionality is best coded via
> subprocess.Popen(). I need to read output from these spawned children
> via a pipe from their stdout, hence something like
> 
>      p= subprocess.Popen(args, stdout=subprocess.PIPE)
> 
> This means that somewhere a pipe file descriptor is opened on the
> parent side to read from the child's stdout. When, if ever, is that
> descriptor closed? Per-process FDs are limited and I am looping
> infinitely so I need to be very careful about not running out of them.
> Are there any other FDs related to this operation that also need to be
> closed?
> 
> Testing with the interpreter (2.6, MacOSX) it appears that p.stdout is
> being closed somehow by someone other than me:
> 
> import subprocess
> args= ["echo", "This is a mystery!"]
> i= 0
> while True:
>     p= subprocess.Popen(args, stdout=subprocess.PIPE)
>     for line in p.stdout:
>         print "[%5d] %s" % (i, line.strip())
>     i+= 1
> 
> The above code closes nothing but appears to run indefinitely without
> running the parent out of FDs. WTF is going on here?

Garbage collection is going on here.

On the second and subsequent invocations of:

	p= subprocess.Popen(args, stdout=subprocess.PIPE)

the assignment to p causes its previous value to become unreferenced.

This will eventually result in a call to the __del__ method of the
now-unreferenced Popen object, which will poll to see if the child is
still running.

If it has terminated, the child process will be reaped and the Popen
object can be discarded. Otherwise, it will be put onto a list
(subprocess._active) of processes to be polled by subprocess._cleanup(),
which is called whenever a new Popen object is created.

IOW, Popen objects are garbage-collected in a "sane" manner; the
associated child processes are allowed to continue running and will
be reaped semi-automatically once they terminate (at least, they won't
accumulate without bound).

> Also, does Popen.returncode contain only the child's exit code or is
> does it also contain signal info like the return of os.wait()?

If the child terminated normally, .returncode will be non-negative, and
contains the "exit status", i.e. the value returned from main() or passed
to exit() etc.

If the child terminated on a signal, .returncode will be negative, and
contains the negation of the signal number.

If the child is still running (or is a zombie), .returncode will be None.