On 24. 05. 22 15:47, wenzel.jakob--- via capi-sig wrote:
Hi,
The object layout "contract" of
Py_LIMITED_APIcurrently seems too underspecified for it to be useful for binding tools, and so I welcome this step in making it more usable.Some thoughts:
it's not just extensions inheriting from core Python types, but also the other way around. For example, if I create a new type with
PyType_FromSpecand then extend it within Python, the CPython implementation will assume that it is legal to enlarge__basicsize__and stick in an extra dictionary at the end.Getting this approach to work for
tp_itemsizesounds a non-trivial technical problem to me. It would complicate the indexing implementation for item access in core types like tuples, lists, etc (where the resulting performance cost is likely not acceptable). My suggestion would be to require that subclasses can't changetp_itemsize.
Oh, I don't intend to remove the tp_itemsize inheritance caveat:
If the base type has a non-zero tp_itemsize, it is generally not safe to set tp_itemsize to a different non-zero value in a subtype (though this depends on the implementation of the base type).
– https://docs.python.org/3/c-api/typeobj.html?highlight=tp_itemsize#c.PyTypeO...
I do intend to make this safe if all bases use PyObject_GetTypeItemData
for accessing per-item data, but I don't intend to make tuple or
list do that.
- Did you mean
PyObject_GetTypeSize(you wrotePyObject_GetTypeataSizewhich sounded like a typo).
PyObject_GetTypeDataSize, missing D
None of the names are final, of course.
- The
PyType_FromMetaclassPR would go nicely along with these changes. Their combination would legitimize what is already happening in practice (e.g in PySide).
Yup.
Thanks, Wenzel
Petr Viktorin wrote:
Hello, When reviewing the PyType_FromMetaclass function proposed in PR 93012, it occurred to me that it would be good to add better support for extending types with opaque structs. Consider the tutorial example of extending list, adapted to PyType_Spec: typedef struct { PyListObject list; int state; } SubListObject;
static PyType_Spec Sublist_spec = { .name = "sublist.SubList", .basicsize = sizeof(SubListObject), ... };
If the PyListObject struct is not available (or opaque), as in the stable ABI, this approach won't work. The practical issue in PR 93012 is with metaclasses, which extend PyTypeObject or PyHeapTypeObject. Same idea but a bit “too meta” to be a good example. “Binding generator”-type projects that use the limited API resort to hacks in this area: see PySide or the experimental limited-API branch of nanobind. I propose adding API that treats the subclass-specific data as a separate struct, and the “base” as an opaque blob (manipulated either by accessor functions or, if the struct is available, directly). Concretely:
PyType_Spec.basicsize can be a negative (or zero) N to request -N
bytes of *extra* storage on top of what the base class needs, so that the PyType_Spec can be static & read-only.
a new void* PyObject_GetTypeData(PyObject *obj, PyTypeObject *cls)
returns the pointer to data specific to cls.
a new Py_ssize_t PyObject_GetTypeataSize(PyTypeObject *cls) returns
the size of that data (computed as cls->tp_basicsize - cls->tp_base->tp_basicsize).
itemsize would get similar treatment, but I'll leave details to the PEP
Intended usage: typedef struct { int state; } SubListObject;
static PyType_Spec Sublist_spec = { .name = "sublist.SubList", .basicsize = -sizeof(SubListObject), ... };
...
sublist_type = PyType_FromSpec(Sublist_spec);
...
SubListObject *data = PyObject_GetTypeData(instance, sublist_type); data->state++;
What do you think? Is it a PEP-worthy idea?
capi-sig mailing list -- capi-sig@python.org To unsubscribe send an email to capi-sig-leave@python.org https://mail.python.org/mailman3/lists/capi-sig.python.org/ Member address: encukou@gmail.com