repeat an array without allocation
Hi all, is there an efficient way to do the following without allocating A where A = np.repeat(x, [4, 2, 1, 3], axis=0) c = A.dot(b) # b.shape thanks -- srean
nope; its impossible to express A as a strided view on x, for the repeats you have. even if you had uniform repeats, it still would not work. that would make it easy to add an extra axis to x without a new allocation; but reshaping/merging that axis with axis=0 would again trigger a copy, as it would require a non-integer stride. On Mon, May 5, 2014 at 6:34 AM, srean <srean.list@gmail.com> wrote:
Hi all,
is there an efficient way to do the following without allocating A where
A = np.repeat(x, [4, 2, 1, 3], axis=0) c = A.dot(b) # b.shape
thanks -- srean
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Sun, May 4, 2014 at 9:34 PM, srean <srean.list@gmail.com> wrote:
Hi all,
is there an efficient way to do the following without allocating A where
A = np.repeat(x, [4, 2, 1, 3], axis=0) c = A.dot(b) # b.shape
If x is a 2D array you can call repeat **after** dot, not before, which will save you some memory and a few operations:
a = np.random.rand(4, 5) b = np.random.rand(5, 6) np.allclose(np.repeat(a, [4, 2, 1, 3], axis=0).dot(b), ... np.repeat(a.dot(b), [4, 2, 1, 3], axis=0)) True
Similarly, if x is a 1D array, you can sum the corresponding items of b before calling dot:
a = np.random.rand(4) b = np.random.rand(10) idx = np.concatenate(([0], np.cumsum([4,2,1,3])[:-1])) np.allclose(np.dot(np.repeat(a, [4,2,1,3] ,axis=0), b), ... np.dot(a, np.add.reduceat(b, idx))) ... ) True
Jaime -- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.
Great ! thanks. I should have seen that. Is there any way array multiplication (as opposed to matrix multiplication) can be sped up without forming A and (A * b) explicitly. A = np.repeat(x, [4, 2, 1, 3], axis = 0) # A.shape == 10,10 c = sum(b * A, axis = 1) # b.shape == 10,10 In my actual setting b is pretty big, so I would like to avoid creating another array the same size. I would also like to avoid a Python loop. st = 0 for (i,rep) in enumerate([4, 2, 1, 3]): end = st + rep c[st : end] = np.dot(b[st : end, :], a[i,:]) st = end Is Cython the only way ? On Mon, May 5, 2014 at 1:20 AM, Jaime Fernández del Río < jaime.frio@gmail.com> wrote:
On Sun, May 4, 2014 at 9:34 PM, srean <srean.list@gmail.com> wrote:
Hi all,
is there an efficient way to do the following without allocating A where
A = np.repeat(x, [4, 2, 1, 3], axis=0) c = A.dot(b) # b.shape
If x is a 2D array you can call repeat **after** dot, not before, which will save you some memory and a few operations:
a = np.random.rand(4, 5) b = np.random.rand(5, 6) np.allclose(np.repeat(a, [4, 2, 1, 3], axis=0).dot(b), ... np.repeat(a.dot(b), [4, 2, 1, 3], axis=0)) True
Similarly, if x is a 1D array, you can sum the corresponding items of b before calling dot:
a = np.random.rand(4) b = np.random.rand(10) idx = np.concatenate(([0], np.cumsum([4,2,1,3])[:-1])) np.allclose(np.dot(np.repeat(a, [4,2,1,3] ,axis=0), b), ... np.dot(a, np.add.reduceat(b, idx))) ... ) True
Jaime
-- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
If b is indeed big I don't see a problem with the python loop, elegance aside; but Cython will not beat it on that front. On Mon, May 5, 2014 at 9:34 AM, srean <srean.list@gmail.com> wrote:
Great ! thanks. I should have seen that.
Is there any way array multiplication (as opposed to matrix multiplication) can be sped up without forming A and (A * b) explicitly.
A = np.repeat(x, [4, 2, 1, 3], axis = 0) # A.shape == 10,10 c = sum(b * A, axis = 1) # b.shape == 10,10
In my actual setting b is pretty big, so I would like to avoid creating another array the same size. I would also like to avoid a Python loop.
st = 0 for (i,rep) in enumerate([4, 2, 1, 3]): end = st + rep c[st : end] = np.dot(b[st : end, :], a[i,:]) st = end
Is Cython the only way ?
On Mon, May 5, 2014 at 1:20 AM, Jaime Fernández del Río < jaime.frio@gmail.com> wrote:
On Sun, May 4, 2014 at 9:34 PM, srean <srean.list@gmail.com> wrote:
Hi all,
is there an efficient way to do the following without allocating A where
A = np.repeat(x, [4, 2, 1, 3], axis=0) c = A.dot(b) # b.shape
If x is a 2D array you can call repeat **after** dot, not before, which will save you some memory and a few operations:
a = np.random.rand(4, 5) b = np.random.rand(5, 6) np.allclose(np.repeat(a, [4, 2, 1, 3], axis=0).dot(b), ... np.repeat(a.dot(b), [4, 2, 1, 3], axis=0)) True
Similarly, if x is a 1D array, you can sum the corresponding items of b before calling dot:
a = np.random.rand(4) b = np.random.rand(10) idx = np.concatenate(([0], np.cumsum([4,2,1,3])[:-1])) np.allclose(np.dot(np.repeat(a, [4,2,1,3] ,axis=0), b), ... np.dot(a, np.add.reduceat(b, idx))) ... ) True
Jaime
-- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
participants (3)
-
Eelco Hoogendoorn -
Jaime Fernández del Río -
srean