[Numpy-discussion] Array concatenation performance

[Skipper Seabold]

On Thu, Jul 15, 2010 at 5:54 AM, John Porter <jporter at cambridgesys.com> wrote:
> Has anyone got any advice about array creation. I've been using numpy
> for a long time and have just noticed something unexpected about array
> concatenation.
>
> It seems that using numpy.array([a,b,c]) is around 20 times slower
> than creating an empty array and adding the individual elements.
>
> Other things that don't work well either:
>    numpy.concatenate([a,b,c]).reshape(3,-1)
>    numpy.concatenate([[a],[b],[c]))
>
> Is there a better way to efficiently create the array ?
>

What was your timing for concatenate?  It wins for me given the shape of a.

In [1]: import numpy as np

In [2]: a = np.arange(1000*1000)

In [3]: timeit b0 = np.array([a,a,a])
1 loops, best of 3: 216 ms per loop

In [4]: timeit b1 = np.empty(((3,)+a.shape)); b1[0]=a;b1[1]=a;b1[2]=a
100 loops, best of 3: 19.3 ms per loop

In [5]: timeit b2 = np.c_[a,a,a].T
10 loops, best of 3: 30.5 ms per loop

In [6]: timeit b3 = np.concatenate([a,a,a]).reshape(3,-1)
100 loops, best of 3: 9.33 ms per loop

Skipper

> See the following snippet:
> ---------------------------------------
> import time
> import numpy as nx
> print 'numpy version', nx.version.version
> t = time.time()
> # test array
> a = nx.arange(1000*1000)
> print 'a ',time.time()-t
> t = time.time()
> # create array in the normal way..
> b0 = nx.array([a,a,a])
> print 'b0',time.time()-t
> t = time.time()
> # create using empty array
> b1 = nx.empty((3,)+(a.shape))
> b1[0] = a
> b1[1] = a
> b1[2] = a
> print 'b1',time.time()-t
> print nx.all((b0==b1))
> -----------------------------------------
> Produces the output:
>   numpy version 1.3.0
>   a  0.0019519329071
>   b0 0.286643981934
>   b1 0.0116579532623
>   equal True
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>