Victor Stinner schrieb am 24.06.20 um 17:40:
My practical problem is how to prevent C extensions accessing the PyFloatObject.ob_fval member directly.
Do extensions really do that in their code? I mean, there *is* a macro for doing exactly this thing, which suggests that users should exactly *not* do it themselves but use the macro. I would simply say that anyone accessing the structure fields directly instead of using the intended macro is simply on their own with that choice. If their code breaks, they'll have to fix it in the way that was intended for the last 23 years (I looked that up).
I don't have any data, but to me, this sounds like a non-issue to start with.
In my tests, I renamed PyObject members. For example, rename PyObject.ob_type to PyObject._ob_type, and update Py_TYPE() and Py_SET_TYPE(). If a C function accesses directly PyObject.ob_type, a compilation error is issued.
I think the path of - making macros / (inline) functions available for all use cases - making them available in a backport header file - telling people to use those instead of direct struct access
is the right way. If/when we notice in the future that we need to change an object struct, and macros are available for the use cases that we break (or can be made available during a suitable deprecation phase), then extension authors will notice at that point that they will have to switch to the macros instead of doing whatever breaks for them (or not).
One option would be to have a different stricter build mode where PyFloat_AS_DOUBLE() becomes a function call. Example:
#ifndef Py_LIMITED_API # ifdef OPAQUE_STRUCTURE # define PyFloat_AS_DOUBLE(op) PyFloat_AsDouble(op) # else # define PyFloat_AS_DOUBLE(op) (((PyFloatObject *)(op))->ob_fval) # endif #endif
I think that's too broad. Why make all structs opaque, when we don't even know which ones we may want to touch in the future at all? And, who would really use this mode?
Or maybe it's time to extend the limited C API: add PyFloat_AS_DOUBLE() macro as a function call. Extending the limited C API has multiple advantages:
- It eases the transition of C extensions to the limited C API
- Py_LIMITED_API already exists, there is no need to add yet another
build mode or any new macro
- Most structures are *already* opaque in the limited C API.
We will have to grow it anyway, so why not. We could also add yet another optional header file that adds everything from the full C-API that we can somehow map to the limited C-API, as macros or inline functions. In the worst case, we could still implement a missing function as a lookup and call through a Python object method, if there's no other way to do it in the limited C-API.
In the end, this could lead to a "full C-API wrapper", implemented on top of the limited C-API. Sounds like a good way to port existing code.
The question becomes: how to promote the limited C API? Should it become the default, rather than an opt-in option?
With the above "full wrapper", it could become the default. That would give authors three choices:
- the full C-API (being tied to a minor release) - the limited C-API (limited but providing forward compatibility) - the wrapper (being slower but providing forward+backward compatibility)
Stefan