Greg Ewing schrieb am 01.02.22 um 23:33:
On 2/02/22 8:48 am, Guido van Rossum wrote:
It seems to me that a big part of the problem is that Cython feels entitled to use arbitrary CPython internals.
I think the reason for this is that Cython is trying to be two things at once: (1) an interface between Python and C, (2) a compiler that turns Python code into fast C code.
To address this there could be an option to choose between "compatible code" and "fast code", with the former restricting itself to the stable API.
There is even more than such an option. We use a relatively large set of feature flags that allow us to turn the usage of certain implementation details of the C-API on and off. We use this to adapt to different Python C-API implementations (currently CPython, PyPy, GraalPython and the Limited C-API), although with different levels of support and reliability.
Here's the complete list of feature sets for the different targets:
This can also be used to enable and disable certain dependencies on CPython implementation details, e.g. PyList, PyLong or PyUnicode, but also type specs versus PyTypeObject structs.
Most of these feature flags can be disabled by users. There is no hard guarantee that this always works, because it's impossible to test all combinations, and then there are bugs as well, but most of the flags are independent, which should usually allow to disable them independently.
So, one of the tools that we have in our sleeves when it comes to supporting new CPython versions is also to selectively disable the dependency on a certain C-API feature that changed, at least until we have a way to adapt to the change itself.
In the specific case of the "exc_info" changes, however, that didn't quite work, because that change was really not anticipated at that level of impact. But there is an implementation for Cython 3.0 alpha now, and we'll eventually have a legacy 0.29.x release out that will also adapt in one way or another. Just takes a bit more time.