Descriptor leak in python 2.4 subprocess module

[Michel Lespinasse]

On Thu, Aug 28, 2008 at 10:37:48AM +0100, Tim Golden wrote:
> Michel Lespinasse wrote:
> >I hit an issue with the following python code:
> >    try:
> >        get_orient = subprocess.Popen (['jpegexiforient', '-n', pathfull],
> >                                       stdin = subprocess.PIPE,
> >                                       stdout = subprocess.PIPE)
> >        orient = get_orient.communicate ()[0]
> >    except:
> >        orient = None
> >
> >The application worked fine on my devel machine but I noticed that on a
> >different host, it crashed due to running out of file descriptors.
> >After investigation I found out that the above code was responsible,
> >leaking two file descriptors per invocation if jpegexiforient is not
> >installed on the host.
> 
> This looks like a duplicate of http://bugs.python.org/issue3210.
> Can you confirm if this seems likely (and, if so, perhaps add
> a note to the bug indicating that the problem is on Linux as well)?

The root cause is different (subprocess has separate implementations
for windows and posix), but the issues are similar. In the posix case,
I only observed the issue when you using subprocess.PIPE

The attached patch against subprocess.py (as shipped on debian etch,
based on python version 2.4.4) seems to work for me:

--- /usr/lib/python2.4/subprocess.py	2008-04-16 10:59:00.000000000 -0700
+++ subprocess.py	2008-08-28 13:09:50.000000000 -0700
@@ -970,6 +970,12 @@
             data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB
             os.close(errpipe_read)
             if data != "":
+                if p2cwrite:
+                    os.close(p2cwrite)
+                if c2pread and c2pread not in (p2cwrite,):
+                    os.close(c2pread)
+                if errread and errread not in (p2cwrite, c2pread):
+                    os.close(errread)
                 os.waitpid(self.pid, 0)
                 child_exception = pickle.loads(data)
                 raise child_exception

I've not tested it much, only wrote it a few minutes ago after looking at
the bug you mentionned made me realize that the subprocess module is
actually written in python so I could try and fix it :)

Hope this helps,

-- 
Michel "Walken" Lespinasse
A program is never fully debugged until the last user dies.