Re: [Pandas-dev] [Numpy-discussion] Transonic Vision: unifying Python-Numpy accelerators
Dear Pandas developers, Ralf Gommers wrote me that there was a discussion on the pandas-dev mailing list a couple of weeks ago about adopting Numba as a dependency. We recently wrote a serious text on this subject: http://tiny.cc/transonic-vision. As a side remark, we also played with Transonic and Pandas in this notebook https://github.com/fluiddyn/transonic-demos/blob/master/pandas.ipynb (the binder link to run the benchmarks: https://mybinder.org/v2/gh/fluiddyn/transonic-demos/master)
Date: Wed, 6 Nov 2019 23:49:08 -0500 From: Ralf Gommers <ralf.gommers@gmail.com> To: Discussion of Numerical Python <numpy-discussion@python.org> Subject: Re: [Numpy-discussion] Transonic Vision: unifying Python-Numpy accelerators Message-ID: <CABL7CQhr6ps25Yrp_pAF8vWVLS3_2=Mw4bCOXb0H1dvU9AjTMw@mail.gmail.com> Content-Type: text/plain; charset="utf-8"
On Mon, Nov 4, 2019 at 4:54 PM PIERRE AUGIER < pierre.augier@univ-grenoble-alpes.fr> wrote:
Dear Python-Numpy community,
Transonic is a pure Python package to easily accelerate modern Python-Numpy code with different accelerators (currently Cython, Pythran and Numba).
I'm trying to get some funding for this project. The related work would benefit in particular to Cython, Numba, Pythran and Xtensor.
To obtain this funding, we really need some feedback from some people knowing the subject of performance with Python-Numpy code.
That's one of the reason why we wrote this long and serious text on Transonic Vision: http://tiny.cc/transonic-vision. We describe some issues (perf for numerical kernels, incompatible accelerators, community split between experts and simple users, ...) and possible improvements.
Thanks Pierre, that's a very interesting vision paper.
In case you haven't seen it, there was a discussion on the pandas-dev mailing list a couple of weeks ago about adopting Numba as a dependency (and issues with that).
Your comment on my assessment from 1.5 years ago being a little unfair to Pythran may be true - not sure it was at the time, but Pythran seems to mature nicely.
The ability to switch between just-in-time and ahead-of-time compilation is nice. One thing I noticed is that this actual switching is not completely fluent: the jit and boost decorators have different signatures, and there's no way to globally switch behavior (say with an env var, as for backend selection).
Help would be very much appreciated.
I'd be interested to help think about adoption and/or funding.
Cheers, Ralf
Now a coding riddle:
import numpy as np from transonic import jit
@jit(native=True, xsimd=True) def fxfy(ft, fn, theta): sin_theta = np.sin(theta) cos_theta = np.cos(theta) fx = cos_theta * ft - sin_theta * fn fy = sin_theta * ft + cos_theta * fn return fx, fy
@jit(native=True, xsimd=True) def fxfy_loops(ft, fn, theta): n0 = theta.size fx = np.empty_like(ft) fy = np.empty_like(fn) for index in range(n0): sin_theta = np.sin(theta[index]) cos_theta = np.cos(theta[index]) fx[index] = cos_theta * ft[index] - sin_theta * fn[index] fy[index] = sin_theta * ft[index] + cos_theta * fn[index] return fx, fy
How can be compared the performances of these functions with pure Numpy, Numba and Pythran ?
You can find out the answer in our note http://tiny.cc/transonic-vision :-)
Pierre
Message: 1 Date: Thu, 31 Oct 2019 21:16:06 +0100 (CET) From: PIERRE AUGIER <pierre.augier@univ-grenoble-alpes.fr> To: numpy-discussion@python.org Subject: [Numpy-discussion] Transonic Vision: unifying Python-Numpy accelerators Message-ID: < 1080118635.5930814.1572552966711.JavaMail.zimbra@univ-grenoble-alpes.fr>
Content-Type: text/plain; charset=utf-8
Dear Python-Numpy community,
Few years ago I started to use a lot Python and Numpy for science. I'd like to thanks all people who contribute to this fantastic community.
I used a lot Cython, Pythran and Numba and for the FluidDyn project, we created Transonic, a pure Python package to easily accelerate modern Python-Numpy code with different accelerators. We wrote a long and serious text to explain why we think Transonic could have a positive impact on the scientific Python ecosystem.
Here it is: http://tiny.cc/transonic-vision
Feedback and discussions would be greatly appreciated!
Pierre
-- Pierre Augier - CR CNRS http://www.legi.grenoble-inp.fr LEGI (UMR 5519) Laboratoire des Ecoulements Geophysiques et Industriels BP53, 38041 Grenoble Cedex, France tel:+33.4.56.52.86.16
On Sat, Nov 16, 2019 at 1:49 AM PIERRE AUGIER < pierre.augier@univ-grenoble-alpes.fr> wrote:
Dear Pandas developers,
Ralf Gommers wrote me that there was a discussion on the pandas-dev mailing list a couple of weeks ago about adopting Numba as a dependency.
We recently wrote a serious text on this subject: http://tiny.cc/transonic-vision.
As a side remark, we also played with Transonic and Pandas in this notebook https://github.com/fluiddyn/transonic-demos/blob/master/pandas.ipynb (the binder link to run the benchmarks: https://mybinder.org/v2/gh/fluiddyn/transonic-demos/master)
Thanks for sharing Pierre. For those who are interested, here are the results of running those benchmarks: Cython: 133 ms ± 11 ms Numba: 39.3 ms ± 5.52 ms Pythran: 36.7 ms ± 2.43 ms Cython + type annotations: 56.3 ms ± 6.56 ms The ease of switching between Cython, Numba and Pythran with Transonic in your notebook is very cool. Cheers, Ralf
Date: Wed, 6 Nov 2019 23:49:08 -0500 From: Ralf Gommers <ralf.gommers@gmail.com> To: Discussion of Numerical Python <numpy-discussion@python.org> Subject: Re: [Numpy-discussion] Transonic Vision: unifying Python-Numpy accelerators Message-ID: <CABL7CQhr6ps25Yrp_pAF8vWVLS3_2= Mw4bCOXb0H1dvU9AjTMw@mail.gmail.com> Content-Type: text/plain; charset="utf-8"
On Mon, Nov 4, 2019 at 4:54 PM PIERRE AUGIER < pierre.augier@univ-grenoble-alpes.fr> wrote:
Dear Python-Numpy community,
Transonic is a pure Python package to easily accelerate modern Python-Numpy code with different accelerators (currently Cython, Pythran and Numba).
I'm trying to get some funding for this project. The related work would benefit in particular to Cython, Numba, Pythran and Xtensor.
To obtain this funding, we really need some feedback from some people knowing the subject of performance with Python-Numpy code.
That's one of the reason why we wrote this long and serious text on Transonic Vision: http://tiny.cc/transonic-vision. We describe some issues (perf for numerical kernels, incompatible accelerators, community split between experts and simple users, ...) and possible improvements.
Thanks Pierre, that's a very interesting vision paper.
In case you haven't seen it, there was a discussion on the pandas-dev mailing list a couple of weeks ago about adopting Numba as a dependency (and issues with that).
Your comment on my assessment from 1.5 years ago being a little unfair to Pythran may be true - not sure it was at the time, but Pythran seems to mature nicely.
The ability to switch between just-in-time and ahead-of-time compilation is nice. One thing I noticed is that this actual switching is not completely fluent: the jit and boost decorators have different signatures, and there's no way to globally switch behavior (say with an env var, as for backend selection).
Help would be very much appreciated.
I'd be interested to help think about adoption and/or funding.
Cheers, Ralf
Now a coding riddle:
import numpy as np from transonic import jit
@jit(native=True, xsimd=True) def fxfy(ft, fn, theta): sin_theta = np.sin(theta) cos_theta = np.cos(theta) fx = cos_theta * ft - sin_theta * fn fy = sin_theta * ft + cos_theta * fn return fx, fy
@jit(native=True, xsimd=True) def fxfy_loops(ft, fn, theta): n0 = theta.size fx = np.empty_like(ft) fy = np.empty_like(fn) for index in range(n0): sin_theta = np.sin(theta[index]) cos_theta = np.cos(theta[index]) fx[index] = cos_theta * ft[index] - sin_theta * fn[index] fy[index] = sin_theta * ft[index] + cos_theta * fn[index] return fx, fy
How can be compared the performances of these functions with pure Numpy, Numba and Pythran ?
You can find out the answer in our note http://tiny.cc/transonic-vision :-)
Pierre
Message: 1 Date: Thu, 31 Oct 2019 21:16:06 +0100 (CET) From: PIERRE AUGIER <pierre.augier@univ-grenoble-alpes.fr> To: numpy-discussion@python.org Subject: [Numpy-discussion] Transonic Vision: unifying Python-Numpy accelerators Message-ID: < 1080118635.5930814.1572552966711.JavaMail.zimbra@univ-grenoble-alpes.fr
Content-Type: text/plain; charset=utf-8
Dear Python-Numpy community,
Few years ago I started to use a lot Python and Numpy for science. I'd
like to
thanks all people who contribute to this fantastic community.
I used a lot Cython, Pythran and Numba and for the FluidDyn project, we created Transonic, a pure Python package to easily accelerate modern Python-Numpy code with different accelerators. We wrote a long and serious text to explain why we think Transonic could have a positive impact on the scientific Python ecosystem.
Here it is: http://tiny.cc/transonic-vision
Feedback and discussions would be greatly appreciated!
Pierre
-- Pierre Augier - CR CNRS http://www.legi.grenoble-inp.fr LEGI (UMR 5519) Laboratoire des Ecoulements Geophysiques et Industriels BP53, 38041 Grenoble Cedex, France tel:+33.4.56.52.86.16
Pandas-dev mailing list Pandas-dev@python.org https://mail.python.org/mailman/listinfo/pandas-dev
participants (2)
-
PIERRE AUGIER -
Ralf Gommers