On 2020-06-09 18:23, Ronald Oussoren wrote:
On 9 Jun 2020, at 16:56, Petr Viktorin <encukou@gmail.com> wrote:
Hello, Now that PEP 573 (Module State Access from C Extension Methods) is in, it's time to address the most important part missing from it: module state access from slot methods (like tp_init or nb_add).
This has long history; see e.g. a thread I started in 2015: https://mail.python.org/pipermail/import-sig/2015-July/thread.html#1037
Most recently, a possible solution, an MRO walker, was one of the last things removed from PEP 573 before acceptance, because the proposed solution was broken: see https://mail.python.org/archives/list/python-dev@python.org/message/PFBAQWQA...
Here's the removed text:
Slot methods
To allow access to
per-module state
_ from slot methods, an MRO walker will be implemented:: PyTypeObject *PyType_DefiningTypeFromSlotFunc(PyTypeObject *type, int slot, void *func) The walker will go through bases of heap-allocatedtype
and search for class that definesfunc
at itsslot
. Thefunc
does not need to be inherited bytype
(i.e. it may have been overridden in a subclass). The only requirement for the walker to find the defining class is that the defining class must be heap-allocated. On failure, exception is set and NULL is returned.and the last iteration of the implementation is here: https://github.com/Dormouse759/cpython/blob/41fe314c5124fa9434d56169868dfc03...
A MRO walker is not efficient.
Not only that, the MRO walker is not necessarily correct, in particular when multiple types on the MRO uses the same C function to implement a slot.
I'm not *that* worried about this. The module's author should be in control of how the C functions are used.
The other viable options I'm aware of are:
- putting a pointer in the class object (which would need reliable class-level storage, see https://mail.python.org/archives/list/capi-sig@python.org/thread/BWVQLET4L3W... )
- solutions that aren't fully general, like context variables or putting a pointer in each instance (thorny because constructors/initializers are themselves slots)
- "__typeslots__" mentioned in https://mail.python.org/pipermail/import-sig/2015-July/001035.html -- which are, IMO, quite invasive
Wouldn’t it be possible to pass the class as an additional argument to the slot function? That requires changing the function signature for slots to use the information, but AFAIK the extra argument wouldn’t be an ABI break in practice if the extra argument is added to the end of the signature (if it would technically be undefined behaviour).
Unfortunately, no. We're not in control of all code that calls the slot
methods. Anyone can call PyType_GetSlot(Py_TYPE(o), Py_tp_iter)(o)
.