[SciPy-User] Efficiently applying a function over a large array

[Jose Gomez-Dans]

David, thanks for your reply

On 9 February 2010 16:42, David Warde-Farley <dwf at cs.toronto.edu> wrote:

>
> On 9-Feb-10, at 11:12 AM, Jose Gomez-Dans wrote:
>
> > for i in ny:
> >   for j in nx:
> >      Out[i,j] = MyFunc ( arr1[i,j,:], arr2[i,j,:], arr3[i,j,:] )
> >
> > but the array size is quite large (>1000x1000 elements), so I would
> > like to know what the most efficient way of doing this would be.
>
> You'll have to provide more details about what MyFunc is doing, as how
> efficient you can make it will depend critically on this.
>

It is doing a fair bit of work, but in essence, from the three vectors it
gets, it generates a set of matrices, and performs a number of matrix
operations (sums and products), and a matrix inversion too. What you get out
of the function is a scalar.

Thinking about it, I thought about list comprehensions:

arr1=numpy.ones(25).reshape(5,5)
arr2=numpy.arange(25).reshape(5,5)
x=[[ func(arr1[i0,i1], arr2[i0,i1]) for i1 in xrange(5)] for i0 in
xrange(5)]

This works, but I don't know how well it's going to apply to my case. I
understand that my function might take some time to evaluate, and that's
fine, but I was worried of the looping taking most of the time.

Thanks!
J
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20100209/ee440064/attachment.html>