[Python-Dev] Internal counter to debug leaking file descriptors

[anatoly techtonik]

On Tue, Aug 31, 2010 at 5:03 PM, Guido van Rossum <guido at python.org> wrote:
> If you wanted to do something like this in the Python stdlib, you'd
> have to monkey-patch (with a proxy/wrapper) all places that can open
> or close a filedescriptor -- os.open, os.popen, os.close, file
> open/close, socket open/close, and probably a bunch more that I've
> forgotten. Also some extension modules may open file descriptors
> directly through the C interfaces.

Actually I monkey-patched fdopen and open, but it appeared that it is
not enough. Extensions cause the biggest problem. How can I
monkey-patch opening of file descriptor deep inside mmap module? How
can I know that a file descriptor is opened there at all? I thought
that maybe Python has internal API for opening file descriptors and it
is possible to intercept the operation on this level. Is it feasible
to route all file descriptor open operations through such API that
allows to audit open/close operations and filenames through callback?

> On Linux you can look somewhere in /proc, but I don't know that it
> would help you find where a file was opened.

If I can query FD counter - I can automate the process of walking
through the code line by line to find places where this descriptor
incremented or decremented. Of course it would be nice to get access
to FD stack so that a full filename can also be retrieved in this
case.

It would be nice if at least Linux implementation provided a way to
detect leaking descriptors, thanks for suggestions, but my expertise
and available resources are limited to Windows machines, so for now I
won't be able to try anything more complicated than an
unpack-and-launch Linux solution.
-- 
anatoly t.