[Python-Dev] PEP 442 clarification for type hierarchies

Stefan Behnel stefan_ml at behnel.de
Sun Aug 4 17:59:57 CEST 2013


Stefan Behnel, 04.08.2013 15:24:
> Stefan Behnel, 04.08.2013 09:23:
>> I'm currently catching up on PEP 442, which managed to fly completely below
>> my radar so far. It's a really helpful change that could end up fixing a
>> major usability problem that Cython was suffering from: user provided
>> deallocation code now has a safe execution environment (well, at least in
>> Py3.4+). That makes Cython a prime candidate for testing this, and I've
>> just started to migrate the implementation.
>>
>> One thing that I found to be missing from the PEP is inheritance handling.
>> The current implementation doesn't seem to care about base types at all, so
>> it appears to be the responsibility of the type to call its super type
>> finalisation function. Is that really intended? Couldn't the super type
>> call chain be made a part of the protocol?
>>
>> Another bit is the exception handling. According to the documentation,
>> tp_finalize() is supposed to first save the current exception state, then
>> do the cleanup, then call WriteUnraisable() if necessary, then restore the
>> exception state.
>>
>> http://docs.python.org/3.4/c-api/typeobj.html#PyTypeObject.tp_finalize
>>
>> Is there a reason why this is left to the user implementation, rather than
>> doing it generically right in PyObject_CallFinalizer() ? That would also
>> make it more efficient to call through the super type hierarchy, I guess. I
>> don't see a need to repeat this exception state swapping at each level.
>>
>> So, essentially, I'm wondering whether PyObject_CallFinalizer() couldn't
>> just set up the execution environment and then call all finalisers of the
>> type hierarchy in bottom-up order.
> 
> I continued my implementation and found that calling up the base type
> hierarchy is essentially the same code as calling up the hierarchy for
> tp_dealloc(), so that was easy to adapt to in Cython and is also more
> efficient than a generic loop (because it can usually benefit from
> inlining). So I'm personally ok with leaving the super type calling code to
> the user side, even though manual implementers may not be entirely happy.
> 
> I think it should get explicitly documented how subtypes should deal with a
> tp_finalize() in (one of the) super types. It's not entirely trivial
> because the tp_finalize slot is not guaranteed to be filled for a super
> type IIUC, as opposed to tp_dealloc. I assume the recursive invariant that
> PyType_Ready() copies it would still hold, though.

Hmm, it seems to me by now that the only safe way of handling this is to
let each tp_dealloc() level in the hierarchy call tp_finalize() through
PyObject_CallFinalizerFromDealloc(), instead of calling up the stack in
tp_finalize(). Otherwise, it's a bit fragile for arbitrary tp_dealloc()
functions in base types and subtypes. However, that appears like a rather
cumbersome and inefficient design. It also somewhat counters the advantage
of having a finalisation step before deallocation, if the finalisers are
only called after (partially) cleaning up the subtypes.

ISTM that this feature hasn't been fully thought out...

Stefan




More information about the Python-Dev mailing list