On Tue, Feb 1, 2022 at 4:42 PM Christian Heimes christian@python.org wrote:
I would prefer if we can get Cython and all the other code generator and bindings library off the unstable C-API. They should use the limited API instead. If they require any C-APIs outside the limited API, then we should investigate and figure something out.
I think that everybody (Python and Cython maintainers) agrees on this point, no? Mark Shannon created the issue "[C API] Add explicit support for Cython to the C API" for that: https://bugs.python.org/issue45247
Cython has an experimental support for the limited C API. The problem is that Cython has a long history of using Python internals on purpose to reach best performance. Moreover, on some cases, there was simply no other choice than using directly the Python internals.
There is an on-going effort adding getter and setter functions on two structures which are causing most troubles on Python updates:
* PyThreadState: https://bugs.python.org/issue39947 * PyFrameObject: https://bugs.python.org/issue40421
The goal is to make these two structures opaque as PyInterpreterState (made opaque in Python 3.7). The blocker issue is Cython. I'm one of the volunteer doing this work, but it's a long task, since these structures have many members. It takes time to properly design functions, merge them in Python, modify Cython to use them, get a Cython release, wait packages releases using the new Cython, etc.
--
The problem right now is the pressure put on Cython maintainers to fix Cython as soon as possible. IMO core developers who introduce incompatible changes should be more involved in the Cython changes, since Cython is a **key component** of the Python ecosystem. IMO knowing that a change breaks Cython and relying on "the community" to fix it is not a nice move. Well, that's my opinion ;-)
--
sip is different because it seems like it's only used by PyQt5, and PyQt5 is one of the few projects (with crytography?) using the limited C API ;-)
About SWIG, pybind11, and PyO3, I don't recall seeing any complain on the Python bug tracker about an incompatible change which broke them. I don't doubt that it's the case, but they seem "less critical" for the most popular PyPI projects. It's good to fix them, but it can be done "later", no?
Victor