On Tue, 2020-06-09 at 20:48 +0200, Petr Viktorin wrote:
On 2020-06-09 18:32, Ronald Oussoren wrote:
On 9 Jun 2020, at 14:50, Petr Viktorin <encukou@gmail.com <mailto:encukou@gmail.com>> wrote:
On 2020-06-08 18:00, Sebastian Berg wrote:
Hi, Since there is a discussion about the limited API and headers. A long time ago I asked about this on python-dev: It concerns me a bit that the limited API does not seem to support MetaClasses in PyType_FromSpec (or FromSpecWithBase). In particular, last time I checked, it seemed impossible to extend the type struct within such a metaclass [1]. There also seemed to be an issue with PyType_FromSpecWithBases allocating exactly a base type-object, instead of using
tp_allocasPyType_Type.tp_newwould. This and how to efficiently cache Python objects in methods are the two main concerns I have right now for long-term full adoption of the limited API. (There are other difficulties, but for the ones I am aware, I at least have an idea how to they need to be addressed). I suppose it may actually be possible to hack around the FromSpec issues, although I am not sure right now. unless subclassing PyType_Type in C is for some reason different and very much discouraged? Cheers, Sebastian [1] And e.g. PySide seemed to "do" this, and got away with it, because there is an additional NULL at the end, as a termination for slots or so. Which for them is unused and they just happen to use.I don't think this is about metaclasses per se: you can create a metaclass in the limited API. Rather, it's about attaching state to class objects—at the C level, i.e. allocating some space rather than putting something in the __dict__.
That’s not just a problem for classes, the same is true for subclassing other types for which the instance struct is private (e.g. all builtin types in the limited API).
For classes, the problem is even worse because they're implemented as variable-length objects ("containers", PyVarObject) to hold slots.
For normal objects, you can play tricks with the ob_size of the superclass, which tells you where additional data can be located in the instance. But for classes (and lists, tuples, ...), it's not clear what the memory layout for additional storage should be, let alone how to access/define it.
Is is possible to define a class that uses a metaclass using PyType_FromSpec or one of its variants? AFAIK it is not possible to set the ob_type field of the to be created type using these APIs.
Ah, I see what you mean. No, it's currently not possible to set the metaclass using PyType_FromSpec. I don't see a reason why adding a Py_ob_type slot would be impossible, but I haven't really given it much thought.
Ah, right, I think that was actually one of main issues... For me this was a bit more tedious. I am currently not too scared of using the limited API for the metaclass (in the end, it probably is a true static, although I am not sure that helps with embedding/sub- interpreters). But it would be much better to use (only) the limited API for dynamically creating the metaclasses instances.
I suppose the other issues are indeed identical to the difficulties of subclassing in general (with extended C-struct/data).
The only thing that would worry me, is if anyone thinks that these things should not or cannot be made to work. I don't mind if we don't have concrete plans yet, nothing about this seems quite concrete anyway.
However, I do want to be the person who maneuvers important C-extension modules into a position where sub-interpreters cannot reasonably made to work at *some* point.
So, I assume that it is OK to expect solutions in the future? I.e. by the time we expect large C-extension modules to put effort into working flawlessly in sub-interpreters?
Cheers,
Sebastian
capi-sig mailing list -- capi-sig@python.org To unsubscribe send an email to capi-sig-leave@python.org https://mail.python.org/mailman3/lists/capi-sig.python.org/ Member address: sebastian@sipsolutions.net