Travis Oliphant wrote:
Weave can still help with the "auto-compilation" of the specific library for your type. Ultimately such code will be faster than NumPy can every be.
Yes. weave.blitz() can be used to do the equivalent of this lazy evaluation for you in many cases without much effort. For example: import weave from scipy import arange a = arange(1e7) b = arange(1e7) c=2.0*a+3.0*b # or with weave weave.blitz("c=2.0*a+3.0*b") As Paul D. mentioned.what Tim outlined is essentially template expressions in C++. blitz++ (http://www.oonumerics.org/blitz/) is a C++ template expressions library for array operations, and weave.blitz translates a Numeric expression into C++ blitz code. For the example above on large arrays, you get about a factor of 4 speed up on large arrays. (Notice, the first time you run the example it will be much slower because of compile time. Use timings from subsequent runs.) C:\temp>weave_time.py Expression: c=2.0*a+3.0*b Numeric: 0.678311899322 Weave: 0.162177084984 Speed-up: 4.18253848494 This isn't as good as you can do with hand coded C, but it isn't so bad for the effort involved... I have wished for time to write a weave.f90("c=2.0*a+3.0*b") function because it is very feasible. My guess from simple experimentation is that it would be about as fast as hand coded C for this sort of expression. This might give us another factor of two or three in execution speed and the compile times would come down from tens of seconds to tenths of seconds. Incidentally, the weave calling overhead is large enough to limit its benefit on small arrays. Pat Miller pointed out some ways to get rid of that overhead, and I even wrote some experimental fixes to weave that helped out a lot. Alas, they never were completed fully. Revisiting these would make weave.blitz useful for small arrays as well. Fixing these is probably more work than wrting blitz.f90() All this to say, I think weave basically accomplishes what Tim wants with a different mechanism (letting C++ compilers do the optimization instead of writing this optimization at the python level). It does require a compiler on client machines in its current form (even that can be fixed...), but I think it might prove faster than re-implementing a numeric expression compiler at the python level (though that sounds fun as well). see ya, eric ############################### #weave_time.py ############################### import timeit array_size = 1e7 iterations = 10 setup = """\ import weave from scipy import arange a=arange(%f) b=arange(%f) c=arange(%f) # needed by weave test """ % (array_size, array_size, array_size) expr = "c=2.0*a+3.0*b" print "Expression:", expr numeric_timer = timeit.Timer(expr, setup) numeric_time = numeric_timer.timeit(number=iterations) print "Numeric:", numeric_time/iterations weave_timer = timeit.Timer('weave.blitz("%s")' % expr, setup) weave_timer = timeit.Timer('weave.blitz("%s")' % expr, setup) weave_time = weave_timer.timeit(number=iterations) print "Weave:", weave_time/iterations print "Speed-up:", numeric_time/weave_time
-Travis
------------------------------------------------------- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion