Hi everyone I read the pypy FAQ and some blog posts, which leaves me puzzled and this is why I'm asking here: I'm doing data science and use numpy as only library in the code I'm talking about. A few years ago I used pypy to great success to speed up an algorithm of mine without much development effort. I had to rewrite the code a bit so numpypy was digesting it but it worked great. Now I am trying to do the same thing and numpypy seems to have been dropped in favor of complete support of numpy. In theory this is great - however now my algorithm is at least 4x slower with pypy. Is numpypy is no longer around or is there a way to install it and try the code with it? Is the numpy support by support of the c extension which the JIT cannot optimise? Does this make pypy slower than Cpython? There have been some blog posts about progress to make it faster from 2017 but then nothing. So is there anything I can do? I do not want to write all the numpy stuff I need in plain python as this would take a lot of time and then I would be so close to C, that I could implement everything in C straight away. Best regards Nikolai
I'm really not qualified to address this, but I suspect a lot of these hopes have moved to HPy: https://github.com/hpyproject/hpy On Fri, Oct 8, 2021 at 2:32 PM Werner Heisenberg < werner.heisenberg.g@gmail.com> wrote:
On 9/10/21 12:31 am, Werner Heisenberg wrote:
It turns out the approach in numpypy was not sustainable with the current level of resources available to PyPy. We have a hard enough time trying to catch up with CPython, and are only now releasing a PyPy3.8, when CPython has recently released 3.10. Keeping up with NumPy, with their twice-early release schedule, in addition to the other maintenance tasks for PyPy, was too much, given our very limited budget for ongoing maintenance. We ended up disabling the numpypy extension for python3+ since the code no longer translated. As Dan Stromberg mentioned, currently we view HPy as the way forward: we predict it will allow the calls to C code to be closer to the speed of CPython. Note this is a more modest goal than numpypy, since numpypy could have provided Numba-like performance. I guess the answer to "what can you do" could be anything from - help up move HPy forward - run an effort to revive numpypy (this will have to be a long-term commitment) - get involved in ongoing maintenance of PyPy which would free up other resources. Matti
Hi, HPy becomes somehow usable :-) I investigated what could give HPy with something similar to Numpy (piconumpy, a C extension providing an array class). Some very simple preliminary results: https://github.com/hpyproject/piconumpy/tree/microbench_loop_sum/bench/micro... Just an explicit loop to sum 10000 random floats with PyPy (and Julia for comparison): {'cache_tag': 'pypy37', 'version': sys.pypy_version_info(major=7, minor=3, micro=6, releaselevel='final', serial=0)} list : 1.73e-05 s ( 1.6 * Julia) purepy : 1.97e-05 s ( 1.9 * Julia) numpy : 4.12e-03 s (388.6 * Julia) _piconumpy_hpy : 2.14e-04 s ( 20.2 * Julia) _piconumpy_cpython_capi : 1.19e-03 s (112.1 * Julia) I have no idea how much HPy+PyPy could be improved in terms of perf for such code. Pierre ----- Mail original -----
I'm really not qualified to address this, but I suspect a lot of these hopes have moved to HPy: https://github.com/hpyproject/hpy On Fri, Oct 8, 2021 at 2:32 PM Werner Heisenberg < werner.heisenberg.g@gmail.com> wrote:
On 9/10/21 12:31 am, Werner Heisenberg wrote:
It turns out the approach in numpypy was not sustainable with the current level of resources available to PyPy. We have a hard enough time trying to catch up with CPython, and are only now releasing a PyPy3.8, when CPython has recently released 3.10. Keeping up with NumPy, with their twice-early release schedule, in addition to the other maintenance tasks for PyPy, was too much, given our very limited budget for ongoing maintenance. We ended up disabling the numpypy extension for python3+ since the code no longer translated. As Dan Stromberg mentioned, currently we view HPy as the way forward: we predict it will allow the calls to C code to be closer to the speed of CPython. Note this is a more modest goal than numpypy, since numpypy could have provided Numba-like performance. I guess the answer to "what can you do" could be anything from - help up move HPy forward - run an effort to revive numpypy (this will have to be a long-term commitment) - get involved in ongoing maintenance of PyPy which would free up other resources. Matti
Hi, HPy becomes somehow usable :-) I investigated what could give HPy with something similar to Numpy (piconumpy, a C extension providing an array class). Some very simple preliminary results: https://github.com/hpyproject/piconumpy/tree/microbench_loop_sum/bench/micro... Just an explicit loop to sum 10000 random floats with PyPy (and Julia for comparison): {'cache_tag': 'pypy37', 'version': sys.pypy_version_info(major=7, minor=3, micro=6, releaselevel='final', serial=0)} list : 1.73e-05 s ( 1.6 * Julia) purepy : 1.97e-05 s ( 1.9 * Julia) numpy : 4.12e-03 s (388.6 * Julia) _piconumpy_hpy : 2.14e-04 s ( 20.2 * Julia) _piconumpy_cpython_capi : 1.19e-03 s (112.1 * Julia) I have no idea how much HPy+PyPy could be improved in terms of perf for such code. Pierre ----- Mail original -----
participants (4)
-
Dan Stromberg -
Matti Picus -
PIERRE AUGIER -
Werner Heisenberg