Instance method decorator garbage collection problem

[Peter Otten]

John Reid wrote:

> Hi,
> 
> I've written a decorator that prints exceptions and I'm having some
> trouble with garbage collection.
> 
> My decorator is:
> 
> import sys
> def print_exception_decorator(fn):
>      def decorator(self, *args, **kwds):
>          try:
>              return fn(*args, **kwds)
>          except:
>              print 'Exception:', sys.exc_info()
>              raise
>      return decorator
> 
> 
> 
> The class I want to decorate the methods of is:
> 
> class InstanceCounted(object):
>      "A class that keeps track of how many instances there are."
>      count = 0
>      def __init__(self):
>          InstanceCounted.count += 1
>      def __del__(self):
>          InstanceCounted.count -= 1
> 
> class A(InstanceCounted):
>      "A class that I want to decorate a method on."
>      def __init__(self):
>          super(A, self).__init__()
>          self.method = print_exception_decorator(self.method)
> 
>      def __del__(self):
>          del self.method
> 
>      def method(self):
>          pass
> 
> 
> 
> When I run the following it does not seem like my object 'a' is garbage
> collected:
> 
> print 'Have %d instances' % InstanceCounted.count
> print 'Creating A'
> a = A()
> print 'Have %d instances' % InstanceCounted.count
> print 'Deleting A'
> del a
> print 'Have %d instances' % InstanceCounted.count
> 
> 
> This is the output:
> 
> Have 0 instances
> Creating A
> Have 1 instances
> Deleting A
> Have 1 instances
> 
> 
> The InstanceCounted.count is 1 at the end. If I omit the call to
> "self.method = print_exception_decorator(self.method)" then the instance
> count goes down to 0 as desired. I thought that the decorator might be
> holding a reference to the instance through the bound method, so I added
>   the __del__() but it doesn't fix the problem.
> 
> Can anyone suggest anything? Is my technique to decorate bound methods
> not a good one? How else should I decorate a bound method?

The problem is that cyclic garbage collection cannot cope with __del__() 
methods. Quoting http://docs.python.org/library/gc.html#gc.garbage

"""
Objects that have __del__() methods and are part of a reference cycle cause 
the entire reference cycle to be uncollectable
"""

The best workaround is to control the object's lifetime explicitly by 
turning it into a context manager, see

http://docs.python.org/reference/datamodel.html#with-statement-context-
managers

or providing a close() method and use contextlib:

>>> class A(object):
...     def close(self):
...             print "break cycles here"
...
>>> class A(object):
...     def __init__(self):
...             self.self = self
...     def __del__(self):
...             print "bye"
...     def close(self):
...             print "breaking cycles here"
...             del self.self
...
>>> a = A()
>>> del a
>>> import contextlib
>>> with contextlib.closing(A()) as b:
...     print "using b"
...
using b
breaking cycles here
>>> del b
bye

Peter