[pypy-issue] [issue828] subprocess on nightly builds, linux 64

Da_Blitz tracker at bugs.pypy.org
Sat Aug 13 16:07:28 CEST 2011

Da_Blitz <pypy at pocketnix.org> added the comment:

Subprocess appears to leak FD's if stdin, stdout or stderr are opened with
subprocess.PIPE (Popen("dsa", stdin=PIPE))

waf uses subprocess in the above manner,
http://code.google.com/p/waf/source/browse/trunk/waflib/Context.py line 331

to replicate run the code below, note i have caught the exception so you can cd
/proc/<printed pid>/fd and have a look at the file descriptors

from subprocess import Popen, PIPE
from time import sleep

    for i in range(10000):
        Popen(["ls"], stdin=PIPE, stdout=PIPE, stderr=PIPE)
    print os.getpid()

ethier you manually have to close the pipes (perhaps introduce a .close to
subprocess) or run the gc to cause the FD's to be garbage collected. 

TL;DR: subprocesess relies on refcounting to clean up open files in a timly manner

perhaps wait could be extended to close FD's when done and advise that all
programs should be waited on, to me this is simmilar to "you should be closing
files" for open files - good practice but no one is doing it on python until
they try out pypy

nosy: +dablitz
status: unread -> chatting

PyPy bug tracker <tracker at bugs.pypy.org>

More information about the pypy-issue mailing list