[Python-ideas] A resources tracker ?

[Andrew Barnert]

On Mar 18, 2014, at 12:57, Tarek Ziadé <tarek at ziade.org> wrote:

> def __allocate_resource(fd) => records the file descriptor that was
> allocated, along with the current traceback.
> def __free_resource(fd) => removes the fd from the list.
> def __is_resource_allocated(fd) => tell if the resource is in the list

I like the general idea. I've actually written wrappers (in Python 2.x and other languages like C++, never Python 3, but similar idea...) to track these kinds of leaks myself.

I don't think you need all of these methods. Just the first one will do it: if you get to __del__ without a close and emit a ResourceWarning, use the info stashed by the allocate function; otherwise, it never gets looked at. And I'm not sure it needs to be a method after all.

What about cases where the io object doesn't actually allocate anything (because you got it from an fd or a socket object or similar?), but you still need to call close. Don't you want the traceback in those cases too? Also, in the most common cases (like open), you're actually creating a chain of two or three objects; do all of them need to store this info?

I don't think you want this on all the time... But when exactly _do_ you want it on? Debug mode? If the warning is enabled at allocation time? A global flag on the io module?

Also, storing an actual traceback is probably a bad idea, as it keeps all kinds of things alive for a very long time. Maybe storing a string representation, or enough info to generate such a representation in the ResourceWarning?

Also, I suspect that knowing the arguments used to allocate the resource (like the filename passed to open or the object's constructor, in the most common case, but also things like the cloexec flag when debugging multi-process apps) might be at least as useful as the traceback, so you might as well add that too.

Anyway, there's pure Python code in Lib/io.py that wraps up the C code. To experiment with this without getting into the C stuff, I think you could edit it to use your own classes that wrap _io.BufferedWriter, etc. instead of just exporting those directly.