[Numpy-discussion] Using multiprocessing (shared memory) with numpy array multiplication

[Brandt Belson]

Unfortunately I can't flatten the arrays. I'm writing a library where the
user supplies an inner product function for two generic objects, and almost
always the inner product function does large array multiplications at some
point. The library doesn't get to know about the underlying arrays.
Thanks,
Brandt



> Message: 2
> Date: Fri, 10 Jun 2011 09:23:10 -0400
> From: Olivier Delalleau <shish at keba.be>
> Subject: Re: [Numpy-discussion] Using multiprocessing (shared memory)
>        with numpy array multiplication
> To: Discussion of Numerical Python <numpy-discussion at scipy.org>
> Message-ID: <BANLkTikjppC90yE56T1mr+byAxXAw32YJA at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> It may not work for you depending on your specific problem constraints, but
> if you could flatten the arrays, then it would be a dot, and you could
> maybe
> compute multiple such dot products by storing those flattened arrays into a
> matrix.
>
> -=- Olivier
>
> 2011/6/10 Brandt Belson <bbelson at princeton.edu>
>
> > Hi,
> > Thanks for getting back to me.
> > I'm doing element wise multiplication, basically innerProduct =
> > numpy.sum(array1*array2) where array1 and array2 are, in general,
> > multidimensional. I need to do many of these operations, and I'd like to
> > split up the tasks between the different cores. I'm not using numpy.dot,
> if
> > I'm not mistaken I don't think that would do what I need.
> > Thanks again,
> > Brandt
> >
> >
> > Message: 1
> >> Date: Thu, 09 Jun 2011 13:11:40 -0700
> >> From: Christopher Barker <Chris.Barker at noaa.gov>
> >> Subject: Re: [Numpy-discussion] Using multiprocessing (shared memory)
> >>        with numpy array multiplication
> >> To: Discussion of Numerical Python <numpy-discussion at scipy.org>
> >> Message-ID: <4DF128FC.8000807 at noaa.gov>
> >> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> >>
> >> Not much time, here, but since you got no replies earlier:
> >>
> >>
> >> >      > I'm parallelizing some code I've written using the built in
> >> >     multiprocessing
> >> >      > module. In my application, I need to multiply many large arrays
> >> >     together
> >>
> >> is the matrix multiplication, or element-wise? If matrix, then numpy
> >> should be using LAPACK, which, depending on how its built, could be
> >> using all your cores already. This is heavily dependent on your your
> >> numpy (really the LAPACK it uses0 is built.
> >>
> >> >      > and
> >> >      > sum the resulting product arrays (inner products).
> >>
> >> are you using numpy.dot() for that? If so, then the above applies to
> >> that as well.
> >>
> >> I know I could look at your code to answer these questions, but I
> >> thought this might help.
> >>
> >> -Chris
> >>
> >>
> >>
> >>
> >>
> >> --
> >> Christopher Barker, Ph.D.
> >> Oceanographer
> >>
> >> Emergency Response Division
> >> NOAA/NOS/OR&R            (206) 526-6959   voice
> >> 7600 Sand Point Way NE   (206) 526-6329   fax
> >> Seattle, WA  98115       (206) 526-6317   main reception
> >>
> >> Chris.Barker at noaa.gov
> >>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20110610/d7689cba/attachment.html>