[pypy-dev] New Python/PyPy extension for object oriented numerically intensive codes ?
pierre.augier at univ-grenoble-alpes.fr
Tue Jan 5 08:42:12 EST 2021
I wish you a Happy New Year.
I would be very interested in using PyPy for numerically intensive scientific codes. I conclude from my experiments that PyPy could potentially be a great tool but that it is strongly limited in this area by the lack of some features in Python. For example:
- Standard Python classes and standard Python instances are too dynamic for what we need for high performance. There should be a way to define in Python less dynamic classes and instances.
- There is no specialized container to gather a fixed number of homogeneous objects and when possible to store them efficiently as native arrays of native variables.
- There is no way to locally disable type and bound checks.
I guess we don't have that in Python because it wouldn't be very useful for CPython and because Python has not been designed for performance. However, with efficient interpreters, I think many users would be happy to trade a bit of dynamism for better performance.
I thought about what was missing and it seems to me that it could be provided by a Python/PyPy extension without addition to the Python language. However, Numpy API is IMHO not adapted. Now that Python is (and will be more and more) compared to Julia, I think it becomes necessary to have a good tool to write efficient numerical codes in pure Python style.
I present here https://github.com/paugier/nbabel/blob/master/py/vector.md a possible new extension providing what I think would be needed to express in OOP Python things reasonable in terms of high performance computing.
The detail of the proposed API is of course not very interesting at this point (it is just a dream). I am more interested about the point of view of PyPy developers and PyPy users about (i) the principle of this project (a Python extension to express in OOP Python things easier to be accelerated than standard Python) and (ii) about the technical feasibility of this project: Is it technically possible to extend Python and PyPy to develop such extension and make it very efficient? Which tools should be used? How should it be written?
More information about the pypy-dev