[Numpy-discussion] vstack and hstack performance penalty

Robert Kern robert.kern at gmail.com
Fri Jan 24 11:21:07 EST 2014


On Fri, Jan 24, 2014 at 4:01 PM, Dinesh Vadhia <dineshbvadhia at hotmail.com>
wrote:
>
> If A is very large and B is very small then np.concatenate(A, B) will copy
> B's data over to A which would take less time than the other way around -
is
> that so?

No, neither array is modified in-place. A new array is created and both A
and B are copied into it. The order is largely unimportant.

> Does 'memory order' mean that it depends on sufficient contiguous
> memory being available for B otherwise it will be fragmented or something
> else?

No, the output is never fragmented. numpy arrays may be strided, but never
fragmented arbitrarily to fit into a fragmented address space.

http://docs.scipy.org/doc/numpy/reference/arrays.ndarray.html#internal-memory-layout-of-an-ndarray

The issue is what axis the concatenation happens on. If it's the first axis
(and both inputs are contiguous), then it only takes two memcpy() calls to
copy the data, one for each input, because the regions where they go into
the output are juxtaposed. If you concatenate on one of the other axes,
though, then the memory regions for A and B will be interleaved and you
have to do 2*N memory copies (N being some number depending on the shape).

--
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20140124/e498f6a7/attachment.html>


More information about the NumPy-Discussion mailing list