[Numpy-discussion] Array vectorization in numpy

Carlos Becker carlosbecker at gmail.com
Tue Jul 19 16:35:32 EDT 2011


Thanks Chad for the explanation on those details. I am new to python and I
still have a lot to learn, this was very useful.
Now I get similar results between matlab and numpy when I re-use the memory
allocated for m with 'm -= 0.5'.

However, if I don't, I obtain this 4x penalty with numpy, even with the
8092x8092 array. Would it be possible to do k = m - 0.5 and pre-alllocate k
such that python does not have to waste time on that?

Another interesting case is something like k = m + n + p, which I guess
should be run with numexpr in order to accelerate it.
Regarding operator evaluation, I thought of the same thing, and it is what
would happen in other languages as well if no expression 'templates' ( such
as what the Eigen library uses ).

I will look at numexpr, I hope I can replace all my matlab image processing
and computational needs with python, and interface with C++ code that uses
Eigen if I need an extra speed boost.
Right now I am trying spyder as a debugger, which looks very nice. If a good
debugger is available, it could totally replace matlab/octave for
researchers/engineers/etc for some specific needs.

I will try numexpr now to see the performance gain.

Thanks to all that replied to this topic, it was very useful.


On Tue, Jul 19, 2011 at 10:11 PM, Nadav Horesh <nadavh at visionsense.com>wrote:

> For such expressions you should try numexpr package: It allows the same
> type of optimisation as Matlab does:  run a single loop over the matrix
> elements instead of repetitive loops and intermediate objects creation.
>
>  Nadav
>
> > Besides the matlab/numpy comparison, I think that there is an inherent
> problem with how expressions are handled, in terms of efficiency.
> > For instance, k = (m - 0.5)*0.3 takes 52msec average here (2000x2000
> array), while k = (m - 0.5)*0.3*0.2 takes 0.079, and k = (m -
> 0.5)*0.3*0.2*0.1 > takes 101msec.
> > Placing parentheses around the scalar multipliers shows that it seems to
> have to do with how expressions are handled, is there sometihng that can >be
> done about this so that numpy can deal with expressions rather than single
> operations chained by python itself? Maybe I am missing the point as well.
>
> ----------------------
> Carlos Becker
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20110719/e0f4354c/attachment.html>


More information about the NumPy-Discussion mailing list