[Numpy-discussion] python reduce vs numpy reduce for outer product

[Robert Kern]

On Sat, Sep 26, 2009 at 17:17, Erik Tollerud <erik.tollerud at gmail.com> wrote:
> I'm encountering behavior that I think makes sense, but I'm not sure
> if there's some numpy function I'm unaware of that might speed up this
> operation.
>
> I have a (potentially very long) sequence of vectors, but for
> examples' sake, I'll stick with three: [A,B,C] with lengths na,nb, and
> nc.  To get the result I want, I first reshape them to (na,1,1) ,
> (1,nb,1) and (1,1,nc) and do:
>
>>>>reduce(np.multiply,[A,B,C])
>
> and the result is what I want... The curious thing is that
>
>>>>np.prod.reduce([A,B,C])

I'm sure you mean np.multiply.reduce().

> throws
>
> ValueError: setting an array element with a sequence.
>
> Presumably this is because np.prod.reduce is trying to operate
> elemnt-wise without broadcasting.

No. np.multiply.reduce() is trying to coerce its argument into an
array. You have given it a list with three arrays that do not have the
compatible shapes.

In [1]: a = arange(5).reshape([5,1,1])

In [2]: b = arange(6).reshape([1,6,1])

In [4]: c = arange(7).reshape([1,1,7])

In [5]: array([a,b,c])
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)

/Users/rkern/Downloads/<ipython console> in <module>()

ValueError: setting an array element with a sequence.

>  But is there a way to make the
> ufunc broadcast faster than doing the python-level reduce?  (I tried
> np.prod(broadcast_arrays([A,B,C]),axis=0), but that seemed slower,
> presumably because it needs to allocate the full array for all three
> instead of just once).

Basically yes because it is computing np.array(np.broadcast_arrays([A,B,C])).

> Or, if there's a better way to just start with the first 3 1d
> vectorsand jump straight to the broadcast product (basically, an outer
> product over arbitrary number of dimensions...)?

Well, numpy doesn't support arbitrary numbers of dimensions, nor will
your memory. You won't be able to do more than a handful of dimensions
practically. Exactly what are you trying to do? Specifics, please, not
toy examples.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco