Hi Victor, I wonder if there's a role for HPy in this context? What if instead of evolving the stable ABI and the limited API, instead we were to focus on first-class support for HPy? Surely 5 release cycles would be enough to completely remove the stable ABI and perhaps even the limited API in favor of HPy? Or am I misunderstanding the place of HPy in the ecosystem? --Guido PS. Eric wrote an analyzer for C code and checked it in under Tools/c-analyzer. This currently focuses on finding globals, but perhaps it forms a good starting point for a linter for C extensions? On Tue, Dec 7, 2021 at 3:53 PM Victor Stinner <vstinner@python.org> wrote:
On Tue, Dec 7, 2021 at 3:43 PM Petr Viktorin <encukou@gmail.com> wrote:
If we would deprecate using Py_REFCNT as l-value in the docs, but wait with the conversion until it was *actually* needed, we would not lose anything: (...) ## CPython nogil fork
In CPython, we cannot change structs that are part of the stable ABI -- such as PyObject.ob_refcnt. IMO, getting rid of the macros that access ob_refcnt is a relatively small part of this issue.
The question is if the nogil optimization is relevant for CPython? It's a trade-off between backward compatibility and performance. It seems like some core devs are enthusiast by the idea of getting this work into CPython: count it in ;-) Getting rid of the GIL is an old dream which became reality: the abiltiy of using all CPU cores at the same time!
But not all technical issues have to be solved at once. First, the *API* can be made compatible with nogil: the Python 3.10 Py_RECNT() change made this function compatible with nogil. Later, the question of the stable *ABI* can be studied.
## GraalVM Python
This PEP is not enough to get rid of wrappers in GraalVM, yet it forces users of CPython to adapt.
Currently, it's considered as second class citizen because of the C API issues. The idea is to put GraalVM Python at the same level as CPython.
I would like to put all Python implementation in a fair competition. The competition is great for innovating and should benefit to everybody!
The PEP 674 is part of this goal. You're right that this PEP alone is not enough to remove all wrappers: it doesn't solve all problems.
On the other side, a large PEP like PEP 620 cannot be accepted... because it requires too many changes at once, it's not technically possible to implement all changes in a single Python. It must be done incrementally over multiple Python versions. That's why I'm trying to split the PEP 620 into smaller PEPs (now: PEP 670, PEP 674), write a better rationale for each incremental change, and better study the backward compatibility issues of each incompatible change.
Until disallowing macros as l-values allows concrete improvements in CPython, it should be the job of linters.
Do you know existing linters for C extensions? I don't know any :-(
FWIW, I do encourage alternative implementations to just not support l-value macros. There are only few projects doing this, and the fix is often (but not always!) easy. This should be a very small part of porting something to a different Python implementation (but I could definitely be wrong here, please correct me).
First, PyPy tried to only implement a subset of the C API and promote cffi for incompatible C extensions. This approach failed.
So PyPy decided to be as compatible as possible with CPython for the C API. Now PyPy basically supports the whole C API and doesn't want to drop support for the C extensions "abusing" the C API, like using macros as l-value.
The problem is that supporting all C API "abuses" requires to basically copy everything from CPython and it's inefficient. I didn't mention PyPy in the PEP 674 since this PEP alone doesn't help PyPy to avoid converting efficient PyPy objects to inefficient CPython objects when they are accessed by the C API (PyPy cpyext module).
Victor -- Night gathers, and now my watch begins. It shall not end until my death. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/UFPN4VUV... Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>