[Python-Dev] tracemalloc.get_object_traceback() and "unclosed" ResourceWarning

[Nick Coghlan]

On 29 October 2013 22:07, Victor Stinner <victor.stinner at gmail.com> wrote:
> Hi,
>
> Since Python 3.3, when a file or socket object is destroyed before
> being closed, a ResourceWarning is emitted. The warning is emitted
> where the object is destroyed: it's common to see "gc.collect()"
> location in the Python test suite for example.
>
> Tarek Ziadé asked on Twitter if there is a tool to track the origin of
> the object:
> "I wonder is there's a tool to trace back objects declarations that
> raise ResourceWarning in Py"
> https://twitter.com/tarek_ziade/status/394006754779877377
>
> Nick Coghlan replied that tracemalloc may help:
> "@tarek_ziade tracemalloc (http://www.python.org/dev/peps/pep-0454/ …
> - not accepted yet) will likely make it possible in 3.4, dunno about
> earlier versions."
> https://twitter.com/ncoghlan_dev/status/394010756242350080
>
> I checked the code and it's not so simple. FileIO destructor emits the
> warning (fileio_dealloc_warn() in Modules/_io/fileio.c), but it's not
> possible to "hook" this function. I tried something with weakref, but
> I failed to set my hook.

Ah, I wasn't clear about what I meant (yay, 140 characters!), but I
was also wrong.

I was thinking you could turn on warnings->errors and then catch the
exception, and use that to get hold of the original object that
triggered the resource warning, and then fed *that* into
tracemalloc.get_object_traceback(). But, alas, even if we tweaked
ResourceWarning to make sure that was possible, exceptions from
__del__ methods are squashed by the interpreter, so you're only left
with the text output.

Didn't an earlier version of the PEP have the ability to find a
traceback based on an object's ID rather than the object itself? If
that capability existed (including for snapshots), then an object id
printed in an error message could be matched up with the tracemalloc
output. (ResourceWarning currently only prints the id if the object
repr includes the id, but we could change that to ensure the object id
is always shown).

Anyway, I don't think this feature needs to be included in the PEP,
but it's something we may decide we want to add later.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia