[Python-3000] Removing __del__

[Nick Coghlan]

Giovanni Bajo wrote:
> Raymond Hettinger wrote:
> 
>> In short, __del__ should disappear not because it is useless but
>> because
>> it is hazardous.  The consenting adults philosophy means that we don't
>> put-up artificial barriers to intentional hacks, but it does not mean
>> that we bait the hook and leave error-prone traps for the unwary.  In
>> Py3k, I would like to see explicit finalization as a preferred
>> approach
>> and for weakrefs be the one-way-to-do-it for designs with implicit
>> finalization.
> 
> Raymond, there is one thing I don't understand in your line of reasoning. You
> say that you prefer explicit finalization, but that implicit finalization still
> needs to be supported. And for that, you'd rather drop __del__ and use
> weakrefs. But why? You say that __del__ is harardous, but I can't see how
> weakrefs are less hazardous.

As I see it, __del__ is more hazardous because it's an attractive nuisance - 
it *looks* like it should be easy to use, but I'm willing to bet that a lot of 
the __del__ methods implemented in the wild are either actual or potential 
bugs. For example, it would be easy for a maintenance programmer to make a 
change to include a reference in a data structure from a child node back to 
its parent node to address a problem, and suddenly the application's memory 
usage goes through the roof due to uncollectable cycles. Even the initial 
implementation of the generator __del__ slot in the *Python 2.5 core* was 
buggy, leading to such cycles - if the developers of the Python interpreter 
find it hard to get __del__ right, then there's something seriously wrong with 
it in its current form.

By explicitly stating that __del__ will go away in Py3k, with the current 
intent being to replace it with explicit finalization (via with statements) 
and the implicit finalization offered by weakref callbacks, it encourages 
people to look for ways to make the API for the latter easier to use.

For example, a "finalizer" factory function could be added to weakref:

_finalizer_refs = set()
def finalizer(*args, **kwds):
     """Create a finalizer from an object, callback and keyword dictionary"""
     # Use positional args and a closure to avoid namespace collisions
     obj, callback = args
     def _finalizer(_ref=None):
         """Callable that invokes the finalization callback"""
         # Use closure to get at weakref to allow direct invocation
         # This creates a cycle, so this approach relies on cyclic GC
         # to clean up the finalizer objects!
         try:
             _finalizer_refs.remove(ref)
         except KeyError:
             pass
         else:
             callback(_finalizer)
     # Give callback access to keyword arguments
     _finalizer.__dict__ = kwds
     ref = weakref.ref(obj, _finalizer)
     _finalizer_refs.add(ref)
     return _finalizer

Example usage:

from weakref import finalizer

class Wrapper(object):
     def __init__(self, x=1):
         self._data = finalizer(self, self.finalize, x=x)
     @staticmethod
     def finalize(data):
         print "Finalizing: value=%s!" % data.x
     def get_value(self):
         return self._data.x
     def increment(self, by=1):
         self._data.x += by
     def close(self):
         self._data()  # Explicitly invoke the finalizer
         self._data = None

 >>> test = Wrapper()
 >>> test.get_value()
1
 >>> test.increment(2)
 >>> test.get_value()
3
 >>> del test
Finalizing: value=3!
 >>> test = Wrapper()
 >>> test.get_value()
1
 >>> test.increment(2)
 >>> test.get_value()
3
 >>> test.close()
Finalizing: value=3!
 >>> del test

For comparison, here's the __del__ based version (which has the downside of 
potentially giving the cyclic GC fits if other attributes are added to the 
object):

class Wrapper(object):
     def __init__(self, x=1):
         self._x = x
     def __del__(self):
         if self._x is not None:
             print "Finalizing: value=%s!" % self._x
     def get_value(self):
         return self._x
     def increment(self, by=1):
         self._x += by
     def close(self):
         self.__del__()
         self._x = None

Not counting the import line, both versions are 13 lines long (granted, the 
weakref version would be a bit longer if the finalizer needed access to public 
attributes - in that case, the weakref version would need to use properties to 
hide the existence of the finalizer object).

Cheers,
Nick.

P.S. the central finalizers list also works a treat for debugging why objects 
aren't getting finalized as expected - a simple loop like "for wr in 
weakref.finalizers: print gc.get_referrers(wr)" after a gc.collect() call 
works pretty well.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org