On 2020-06-08 18:00, Sebastian Berg wrote:
Hi,
Since there is a discussion about the limited API and headers. A long time ago I asked about this on python-dev:
It concerns me a bit that the limited API does not seem to support MetaClasses in PyType_FromSpec (or FromSpecWithBase). In particular, last time I checked, it seemed impossible to extend the type struct within such a metaclass [1].
There also seemed to be an issue with PyType_FromSpecWithBases allocating exactly a base type-object, instead of using
tp_alloc
asPyType_Type.tp_new
would.This and how to efficiently cache Python objects in methods are the two main concerns I have right now for long-term full adoption of the limited API. (There are other difficulties, but for the ones I am aware, I at least have an idea how to they need to be addressed).
I suppose it may actually be possible to hack around the FromSpec issues, although I am not sure right now. unless subclassing PyType_Type in C is for some reason different and very much discouraged?
Cheers,
Sebastian
[1] And e.g. PySide seemed to "do" this, and got away with it, because there is an additional NULL at the end, as a termination for slots or so. Which for them is unused and they just happen to use.
I don't think this is about metaclasses per se: you can create a metaclass in the limited API. Rather, it's about attaching state to class objects—at the C level, i.e. allocating some space rather than putting something in the __dict__. AFAIK, calling tp_alloc from PyType_FromSpecWithBases is problematic, because you can't write a custom tp_alloc for a type without assuming too much about the internals.
This all is indeed a problem, and making it work will need some thinking (esp. with inheritance in mind: a superclass and a subclass will need independent storage). But, since it's not needed in Python's standard library, it's not currently on the top of my TODO list. (The thing I want to focus now is documentation, and then access to module state from slot methods.)
And yes, PySide gets away with the hack. That project also reveals more than its share of bugs in CPython, and sometimes needs to be fixed for new Python releases. It's definitely great that PySide exists and works around the issues! But I wouldn't recommend its techniques for general usage.