On 07. 02. 22 14:26, Victor Stinner wrote:
On Mon, Feb 7, 2022 at 2:08 PM Petr Viktorin email@example.com wrote:
Basically, instead of "We'll remove this API now because it prevents moving to a hypothetical moving garbage collector", it should be "Here is a moving garbage collector that speeds Python up by 30%, but to add it we need to remove these 30 deprecated APIs". The deprecation can be proactive, but not the removal.
PEP 674 gives 3 concrete examples of issues already affecting the CPython nogil fork, HPy and GraalPython. They are not hypothetical.
HPy and GraalPython are different projects, and their issues don't affect current users of CPython. It would be great to make an easy-to-use API that would be compatible with HPy and GraalPython. But not at the cost of breaking existing code. Are the advantages of moving to HPy or Graalpython worth porting the code? IMO that's a question each extension author should be able to ask. They shouldn't be forced to do the port by CPython.
As for nogil, that's a promising experiment. If it turns out to be successful, let's remove the parts that are blocking it. Do we already know which parts those are?
CPython is also affected by these issues, but the benefits of PEP 674 (alone) are too indirect, so I chose to avoid mentioning CPython issues directly, to avoid confusion.
IMO, CPython issues are the ones most relevant to CPython. If yo're bringing them up, could you be more specific about them? If we don't discuss them, how do we know if there better ways to solve them than what you're proposing?
It's possible to workaround them: more or less copy/paste CPython inefficient code, as PyPy did years ago. The problem is that the workaround is inefficient and so PyPy cpyext remains slow. Well, HPy address the cpyext performance problem for PyPy and GraalPython ;-)
Hooray! So, the issue is addressed, and we don't need to break code that doesn't care about performance on PyPy! Sounds perfect! More seriously: I expect that there are a lot of extension authors that value API stability over performance on PyPy. They're doing nothing wrong and they don't deserve to be punished for using API they had available (and docs they had available) some years ago.
I don't think that the question is if there is a real problem or not. The question is what's the best migration plan to move existing C extensions towards a better API which don't suffer from these problems.
Once 95% of C extensions will use the limited C API, we would still not be able to change Python internals, because of the 5% remaining C extensions which are stuck at the legacy C API.
And don't forget extensions that use the "bad" parts of limited API, like using Py_TYPE as l-value in Python 3.10-. IMO, the required ABI changes will be more drastic than the API changes.
When we want to change CPython internals, IMO we should know the exact set of APIs that'll need to be removed. Without an actual proposed change, I don't think we can know that. HPy is a good indicator of what API should be deprecated (i.e. where we should find a better way of doing things), but I don't think it gives good reasons for breaking changes.