pypy-dev

Download

pypy-dev@python.org

June 2025

3 participants
1 discussions

Better compatibility of the Python scientific/data stack with fast Python interpreters
by PIERRE AUGIER June 6, 2025

June 6, 2025

Dear Numpy community members and Numpy developers, This email is to get the points of view of the Numpy community members and developers about a subject that I find very important. I'm going to introduce it just in few lines so I write few statements without backing them with proper arguments and without giving links and word definitions. I assume that most people registered to the numpy-discussion list are at least familiar to the subject. I think getting proper compatibility of the Python scientific/data stack with fast Python interpreters is very important for the long-term future of Python for science and data. Nowadays, fast Python implementations like PyPy and GraalPy are - complicated to use (need local builds of wheels) - slow as soon as the code involves Python extensions (because extensions work through an emulation layer, like cpyext in PyPy) Therefore, these fast Python implementations are not as popular as they should be. However, they are really great tools which should be used in particular for scientific/data applications. Unfortunately, if we just follow the pace of the CPython C API evolution, we will basically never have an ecosystem natively compatible with fast Python implementations. The more I think and learn about this subject, the more I think that Numpy has to stop using directly the CPython C API and to be ported to HPy (though there are other alternatives - in particular Nanobind and Cython - that could be discussed). Numpy 1 has been ported to HPy but unfortunately, there have been deep changes with Numpy 2 (and Meson) so it seems that one should restart the porting. Moreover, unfortunately, HPy does not currently receive as much care as it should. It seems to me that the project of fixing the roots of the Python ecosystem has to be relaunched. I think that the dynamics has to come from the Python scientific/data community and in particular Numpy. It is unfortunately outside of the C API working group's scope (see https://discuss.python.org/t/c-api-working-group-and-plan-to-get-a-python-c…). It seems to me that it is necessary (and possible) to get some founding for such an impactful project so that we can get people working on it. I'd like to write a long and serious text on this subject, and I first try to get the points of view of the different people and projects involved. I guess I should write explicit questions: - What do you think about the project of fixing the Python scientific/data stack so that it becomes natively compatible (hence fast and convenient, with easy installations) with alternative and fast Python interpreters? - Do you have points of view on how this should be done, technically (HPy?, something else?) and on other aspects (community, NEP?, founding, ...). - Anything else interesting on this subject? Best regards, Pierre -- Pierre Augier - CR CNRS http://www.legi.grenoble-inp.fr LEGI (UMR 5519) Laboratoire des Ecoulements Geophysiques et Industriels BP53, 38041 Grenoble Cedex, France tel:+33.4.56.52.86.16

7 12