announce: pyjit, a little jit for creating numpy ufuncs
Hi, Inspired by numexpr, pypy and llvm, i've built a simple JIT for creating numpy "ufuncs" (they are not yet real ufuncs). It uses llvm[1] as the backend machine code generator. The main things it can do are: *) parse simple python code (function def's) *) generate SSA assembly code for llvm *) build ufunc code for applying to numpy array's When I say simple I mean it: def calc(a,b): c = (a+b)/2.0 return c No control flow or type inference has been implemented. As with numexpr, significant speedups are possible. I'm putting this announce here to see what the other numpy'ers think. $ svn co http://rubis.rsise.anu.edu.au/local/repos/elefant/pyjit bye, Simon. [1] http://llvm.org/ -- Simon Burton, B.Sc. Licensed PO Box 8066 ANU Canberra 2601 Australia Ph. 61 02 6249 6940 http://arrowtheory.com
Simon Burton <simon@arrowtheory.com> writes:
Hi,
Inspired by numexpr, pypy and llvm, i've built a simple JIT for creating numpy "ufuncs" (they are not yet real ufuncs). It uses llvm[1] as the backend machine code generator.
Cool! I had a look at LLVM, but I wanted something to go into SciPy, and that was too heavy a dependence. However, I could see doing more stuff with this than I can easily with numexpr.
The main things it can do are:
*) parse simple python code (function def's) *) generate SSA assembly code for llvm *) build ufunc code for applying to numpy array's
When I say simple I mean it:
def calc(a,b): c = (a+b)/2.0 return c
No control flow or type inference has been implemented.
As with numexpr, significant speedups are possible.
I'm putting this announce here to see what the other numpy'ers think.
$ svn co http://rubis.rsise.anu.edu.au/local/repos/elefant/pyjit
[1] http://llvm.org/
How do the speedups compare with numexpr? Are there any lessons you learned from this that could apply to numexpr? Could we have a common frontend for numexpr/pyjit, and a different backend for each? Then each wouldn't have to reinvent the wheel in parsing (the same thought goes with weave, too...) I don't have much time to look at it (real life sucking my time :-(), but I'll have a look when I do have the time. -- |>|\/|< /--------------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm@physics.mcmaster.ca
On Mon, Apr 24, 2006 at 04:17:16PM -0400, David M. Cooke wrote:
Simon Burton <simon@arrowtheory.com> writes:
Hi,
Inspired by numexpr, pypy and llvm, i've built a simple JIT for creating numpy "ufuncs" (they are not yet real ufuncs). It uses llvm[1] as the backend machine code generator.
Cool! I had a look at LLVM, but I wanted something to go into SciPy, and that was too heavy a dependence. However, I could see doing more stuff with this than I can easily with numexpr.
Hello, People interested in this might also be interested in PyPy's rctypes and the exploratory work done in PyPy to annotate code using arrays. The goal is "write Python code using numeric arrays and other C libs, then ask PyPy to translate it to C while removing the python wrapper of the C libs, then compile". Then you can run the code as python code when developping and compile the all thing from C to assembly when speed matters. Please note it is a goal. We are not there yet. But any help will be welcome :) -- Nicolas Chauvat logilab.fr - services en informatique avancée et gestion de connaissances
On Mon, 24 Apr 2006 16:17:16 -0400 cookedm@physics.mcmaster.ca (David M. Cooke) wrote:
How do the speedups compare with numexpr?
numexpr segfaults for me (runing timings.py): Program received signal SIGSEGV, Segmentation fault. [Switching to Thread -1209670912 (LWP 31768)] 0xb7d2b696 in PyArray_NewFromDescr (subtype=0x626e6769, descr=0x64007469, nd=1919251557, dims=0x656e696d, strides=0x782d2073, data=0x656c6520, flags=1953391981, obj=0x65736977) at arrayobject.c:3942 3942 arrayobject.c: No such file or directory. in arrayobject.c Simon. -- Simon Burton, B.Sc. Licensed PO Box 8066 ANU Canberra 2601 Australia Ph. 61 02 6249 6940 http://arrowtheory.com
participants (3)
-
cookedm＠physics.mcmaster.ca -
Nicolas Chauvat -
Simon Burton