[Python-3000] Removing __del__

[Michael Chermside]

Giovanni Bajo writes:
> I believe my example is a good use case for __del__ with no good
> enough workaround, which was requested by Micheal in the original post. I
> believe that it would be a mistake to remove __del__ unless we provide a
> graceful alternative (and I don't consider the code above a graceful
> alternative). I still like the __close__ method being proposed.

Thank you!

This is exactly the kind of discussion that I was hoping to engender.
Let me see if I can make the case a little more effectively. First of
all, let clean up Jean-Paul's solution a little bit so it looks prettier
when used. Let's put the following code into a module:

----- deletions.py -----
import weakref

# Maintain a separate list so the weakrefs themselves
# won't be garbage collected.
on_del_callbacks = []


def on_del_invoke(obj, func, *args, **kwargs):
     """This sets up a callback to be executed when an object
     is finalized. It is similar to the old __del__ method but
     without some of the risks and limitations of that method.

     The first argument is an object to watch; the second is a
     callable. After the object being watched gets finalized,
     the callable will be invoked; arguments for this call can
     be provided after the callable.

     Please note that the callable must not be a bound method
     of the object being watched, and the object being watched
     must not be (or be refered to by) one of the arguments
     or else the object will never be garbage collected."""

     def callback(ref):
         on_del_callbacks.remove(ref)
         func(*args, **kwargs)

     on_del_callbacks.append(
         weakref.ref(obj, callback))
--- end deletions.py ---

Performance could be improved in minor ways (avoiding the O(n)
lookup cost in the remove() call; avoiding the need for a
separate function object for each callback; catching obvious
loops and raising an exception immediately to make it more
newbie-friendly), but this will do for discussion.

Using this, your original code:

> class Wrapper:
>     def __init__(self, *args):
>          self.handle = CAPI.init(*args)
>
>     def __del__(self, *args):
>           CAPI.close(self.handle)
>
>     def foo(self):
>           CAPI.foo(self.handle)

becomes this code:

   from deletions import on_del_invoke

   class Wrapper:
       def __init__(self, *args):
           self.handle = CAPI.init(*args)
           on_del_invoke(self, CAPI.close, self.handle)

       def foo(self):
           CAPI.foo(self.handle)

It's actually *fewer* lines this way, and I find it quite
readable. Furthermore, unlike the __del__ version it doesn't
break as soon as someone accidentally puts a Wrapper object
into a loop.

Working from this example, I'm not convinced that the price
of giving up __del__ is really all that high. (But please,
find another example to convince me!)

On the other side of the scales, here are some benefits that
we gain if we get rid of __del__:

   * Simpler GC code which is less likely to have obscure
     bugs that are incredibly difficult to track down. Less
     core developer time spent maintaining complex, fragile
     code.

   * No need to explain about keeping __del__ objects[1] out
     of reference loops. In exchange, we choose explain
     about not passing the object being monitored or
     anything that links to it as arguments to on_del_invoke.
     I find that preferable because: (1) it seems more
     intuitive to me that the callback musn't reference the
     object being finalized, (2) it requires reasoning about
     the call-site, not about all future uses of the object,
     and (3) if the programmer violates this rule then the
     disadvantage is that the objects become immortal -- which
     is true for ALL __del__ objects in loops today.

   * Programmers no longer have the ability to allow __del__
     to resurect the object being finalized. Technically,
     that's a disadvantage, not an advantage, but I honestly
     don't think anyone believes it's a good idea to write
     __del__ methods that resurect the object, so I'm happy
     to lose that ability.

-- Michael Chermside


[1] - I'm using "__del__ object" to mean an object that has
    a __del__ method.