[Python-3000] Removing __del__

[Nick Coghlan]

Giovanni Bajo wrote:
> It would require some effort to make weakref finalizers *barely* as usable
> as __del__, and will absolutely not solve the problem per-se: the user will
> still have to pay attention and understand the hoops (different kind of
> hoops, but still hoops). So, why do we not spend this same time trying to
> *fix* __del__ instead? If somebody comes up with a sane way to define the
> semantic for a new finalizer method (like the __close__ proposal), which can
> be invoked *even* in the case of cycles, would you still prefer to go the
> weakref way?

Yes. I believe any replacement for __del__ should be syntactic sugar for some 
form of weak reference callback. At the moment, we have two finalization 
methods (__del__ and weakref callbacks). Py3k gives us the opportunity to get 
rid of one of them. Since weakref callbacks are strictly more powerful, then 
__del__ should be the one to go.

Having first made the decision to reduce the number of finalization mechanisms 
to exactly one, I then have no problem with the idea of developing an easy to 
use weakref-based approach to replace the current __del__ (which may or may 
not be a magic method).

> So uhm, am I reading it bad or your implementation (like any other similar
> API I have seen till now) create a cycle *just* by using it?

To use Tim's terminology, the weakref (WR) and the callback (CB) are in a 
cycle with each other, so even after CB is invoked and removes WR from the 
global list of finalizers, the two objects won't go away until the next GC 
collection cycle. The weakly referenced object (WO) itself isn't part of the 
cycle and gets finalized at the first opportunity after its reference count 
goes to zero (as shown in my example - the finalizer ran without having to 
call gc.collect() first).

And don't forget that in non-refcounting implementations like Jython, 
IronPython and some flavours of PyPy, even non-cyclic garbage is collected 
through the GC mechanism at an arbitrary time after the last reference is 
released. If you need prompt finalization (for activities such as closing file 
handles or database connections), that's the whole reason the 'with' statement 
was added in Python 2.5.

All that aside, my example finalizer API only took an hour or two to write, 
compared to the significant amount of effort that has gone into the current 
__del__ implementation. There are actually a number of ways to write weakref 
based finalization that avoid that WR-CB cycle I used, but considering the 
trade-offs between those approaches is a lot more than a two-hour project 
(and, not the least bit incidentally, not an assessment I would really want to 
make on my own ;).

> This finalizer
> API ofhuscates user code by forcing to use a separate _data object to hold
> (part of) the context for apparently no good reason, and make the object
> collectable *only* through the cyclic GC (while __del__ would happily be
> invoked in simple cases when the object goes out of context).

It stores part of the context in a separate object for an *excellent* reason - 
it identifies clearly to the Python interpreter *which* parts of the object 
the finalizer can access. The biggest problem with __del__ is that it 
*doesn't* make that distinction, so the interpreter is forced to assume the 
finalizer might touch any part of the object (including the object itself), 
leading to all of the insanity with self-resurrection and the need to relegate 
things to gc.garbage. With a weakref-based approach, you only end up with two 
possible scenarios:

1. Object gets trashed and finalized
2. Object is kept immortal by a strong reference from the callback in the list 
of finalizers

By avoiding teaching people that care about finalization the important 
distinction between "things the finalizer can get at" and "things the object 
can get at but the finalizer can't", you aren't doing them any favours, 
because maintaining that distinction is the easiest way to avoid creating 
uncollectable cycles (i.e. by making sure the finalizer can't get at the other 
objects that might reference back to the current one).

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org