Re: [Numpy-discussion] Meta: too many numerical libraries doing the same thing?
"Paul F. Dubois" wrote:
We had some meetings to discuss using blitz and the truth is that as wrapped by Python there is not much to gain. The efficiency of blitz comes up when you do an array expression in C++. Then x = y + z + w + a + b gets compiled into one loop with no temporary objects created. But this trick is possible because you can bind the assignment. In python you cannot bind the assignment so you cannot do a lazy evaluation of the operations, unless you are willing to go with some sort of function call like x = evaluate(y + z + w). Immediate evaluations means creating temporaries, and performance is dead.
The only gain then would be when you passed a Python-wrapped blitz array back to C++ and did a bunch of operations there.
Personally, I think this could be a big gain. At the moment, if you don't get the performance you need with NumPy, you have to write some of your code in C, and using the Numeric and Python C API is a whole lot of work, particularly if you want your function to work on non-contiguous arrays and/or arrays of any type. I don't know much C++, and I have no idea if Blitz++ fits this bill, but it seemed to me that using an object oriented framework that could take care of reference counting, and allow you to work with generic arrays, and index them naturally, etc, would be a great improvement, even if the performance was the same as the current C API. Perhaps NumPy2 has accomplished that, it sounds like it is a step in the right direction, at least. In a sentence: the most important reason for using a C++ object oriented multi-dimensional array package would be easy of use, not speed. It's nice to hear Blitz++ was considered, it was proably rejected for good reason, but it just looked very promising to me. -Chris -- Christopher Barker, Ph.D. ChrisHBarker@home.net --- --- --- http://members.home.net/barkerlohmann ---@@ -----@@ -----@@ ------@@@ ------@@@ ------@@@ Oil Spill Modeling ------ @ ------ @ ------ @ Water Resources Engineering ------- --------- -------- Coastal and Fluvial Hydrodynamics -------------------------------------- ------------------------------------------------------------------------
In a sentence: the most important reason for using a C++ object oriented multi-dimensional array package would be easy of use, not speed.
It's nice to hear Blitz++ was considered, it was proably rejected for good reason, but it just looked very promising to me.
I believe that Eric's "compiler" module included in SciPy uses Blitz++ to optimize Numeric expressions. You have others who also share your admiration of Blitz++ -Travis
Travis Oliphant wrote:
I believe that Eric's "compiler" module included in SciPy uses Blitz++ to optimize Numeric expressions. You have others who also share your admiration of Blitz++
Yes, it does. That's where I heard about it. That also brings up a good point. Paul mentioned that using something like Blitz++ would only help performance if you could pass it an entire expression, like: x = a+b+c+d. That is exactly what Eric's compiler module does, and it would sure be easier if NumPy already used Blitz++! In Fact, I suppose Eric's compiler is a start towards a tool that could comp9le en entire NumPy function or module. I'd love to be able to just do that (with some tweeking perhaps) rather than having to code it all by hand. My fantasies continue... -Chris -- Christopher Barker, Ph.D. ChrisHBarker@home.net --- --- --- http://members.home.net/barkerlohmann ---@@ -----@@ -----@@ ------@@@ ------@@@ ------@@@ Oil Spill Modeling ------ @ ------ @ ------ @ Water Resources Engineering ------- --------- -------- Coastal and Fluvial Hydrodynamics -------------------------------------- ------------------------------------------------------------------------
participants (2)
-
Chris Barker
-
Travis Oliphant