[Python-ideas] A resources tracker ?

[Tarek Ziadé]

Hey

ResourceWarning instances that are created in some classes' __del__
function, like FileIO
are a great tool to track down a program bad behavior.

In network programming, that's even more important to avoid crashes, or
huge leaks.

But some of them are very hard to fix because we don't get much context,
we just
get warnings at the end of the program execution, when gc.collect is called.

sys:1: ResourceWarning: unclosed file <_io.FileIO name=18 mode='wb'>
sys:1: ResourceWarning: unclosed file <_io.FileIO name=17 mode='rb'>

Here I just know that somewhere, 2 file descriptors where not closed.

What I'd like to be able to do is to track down the origin of those
warnings.

Since __del__ is called asynchronously, it's impossible to track it
right now (or I don't know how)

What we need is a way to keep track of any resource allocation *when it
happens*.

Here's an idea: let's add three private functions in Python's io:

def __allocate_resource(fd) => records the file descriptor that was
allocated, along with the current traceback.
def __free_resource(fd) => removes the fd from the list.
def __is_resource_allocated(fd) => tell if the resource is in the list.

These three functions, plugged in somewhere in io's classes, could be
used in conjunction with ResourceWarning:
when __del__ is called, if the resource was not freed - we'd be able to
know where it was created.

Of course these functions are just a brain dump - I have no idea how io
internals work. But unless I missed it,
something like I've just described is missing in Python.

Cheers
Tarek