[Python-Dev] PEP 442 clarification for type hierarchies

Stefan Behnel stefan_ml at behnel.de
Sun Aug 4 17:59:57 CEST 2013

Stefan Behnel, 04.08.2013 15:24:
> Stefan Behnel, 04.08.2013 09:23:
>> I'm currently catching up on PEP 442, which managed to fly completely below
>> my radar so far. It's a really helpful change that could end up fixing a
>> major usability problem that Cython was suffering from: user provided
>> deallocation code now has a safe execution environment (well, at least in
>> Py3.4+). That makes Cython a prime candidate for testing this, and I've
>> just started to migrate the implementation.
>> One thing that I found to be missing from the PEP is inheritance handling.
>> The current implementation doesn't seem to care about base types at all, so
>> it appears to be the responsibility of the type to call its super type
>> finalisation function. Is that really intended? Couldn't the super type
>> call chain be made a part of the protocol?
>> Another bit is the exception handling. According to the documentation,
>> tp_finalize() is supposed to first save the current exception state, then
>> do the cleanup, then call WriteUnraisable() if necessary, then restore the
>> exception state.
>> http://docs.python.org/3.4/c-api/typeobj.html#PyTypeObject.tp_finalize
>> Is there a reason why this is left to the user implementation, rather than
>> doing it generically right in PyObject_CallFinalizer() ? That would also
>> make it more efficient to call through the super type hierarchy, I guess. I
>> don't see a need to repeat this exception state swapping at each level.
>> So, essentially, I'm wondering whether PyObject_CallFinalizer() couldn't
>> just set up the execution environment and then call all finalisers of the
>> type hierarchy in bottom-up order.
> I continued my implementation and found that calling up the base type
> hierarchy is essentially the same code as calling up the hierarchy for
> tp_dealloc(), so that was easy to adapt to in Cython and is also more
> efficient than a generic loop (because it can usually benefit from
> inlining). So I'm personally ok with leaving the super type calling code to
> the user side, even though manual implementers may not be entirely happy.
> I think it should get explicitly documented how subtypes should deal with a
> tp_finalize() in (one of the) super types. It's not entirely trivial
> because the tp_finalize slot is not guaranteed to be filled for a super
> type IIUC, as opposed to tp_dealloc. I assume the recursive invariant that
> PyType_Ready() copies it would still hold, though.

Hmm, it seems to me by now that the only safe way of handling this is to
let each tp_dealloc() level in the hierarchy call tp_finalize() through
PyObject_CallFinalizerFromDealloc(), instead of calling up the stack in
tp_finalize(). Otherwise, it's a bit fragile for arbitrary tp_dealloc()
functions in base types and subtypes. However, that appears like a rather
cumbersome and inefficient design. It also somewhat counters the advantage
of having a finalisation step before deallocation, if the finalisers are
only called after (partially) cleaning up the subtypes.

ISTM that this feature hasn't been fully thought out...


More information about the Python-Dev mailing list