nicest way to apply an arbitrary sequence of row deltas to an array
Hi everyone, I was wondering if anyone had insight on the best way to solve the following problem. Suppose I have a numpy array called U. U has shape (N,M) Suppose further that I have another array called dU and that dU has shape (P,M) and that P has no particular relationship to N, it could be larger or smaller. Additionally, I have a one dimensional array called idx and idx has shape (P,). Furthermore idx holds integers that are all valid indices into the rows of U. idx almost certainly contains some duplicates. I want, for k in range(P): U[idx[k],:] += dU[k,:] Is there a nice vectorized and efficient way to do this without making the obvious python for loop I included above? For what I am doing, P is usually quite large. I am most interested in a clever use of numpy/scipy commands and Python and not Cython. Thanks in advance for any suggestions. - George
George Dahl <george.dahl <at> gmail.com> writes:
Hi everyone, I was wondering if anyone had insight on the best way to solve the following problem.
Suppose I have a numpy array called U. U has shape (N,M) Suppose further that I have another array called dU and that dU has shape (P,M) and that P has no particular relationship to N, it could be larger or smaller. Additionally, I have a one dimensional array called idx and idx has shape (P,). Furthermore idx holds integers that are all valid indices into the rows of U. idx almost certainly contains some duplicates.
I want,
for k in range(P): U[idx[k],:] += dU[k,:]
Is there a nice vectorized and efficient way to do this without making the obvious python for loop I included above? For what I am doing, P is usually quite large. I am most interested in a clever use of numpy/scipy commands and Python and not Cython.
I'm also interested to see if there are any answers to this; I came across a similar problem recently. It would have been convenient to do something like U[idx] += dU, but this didn't work because there were repeated indices in idx. Here's a short example that shows the problem: In [1]: U = np.array([1., 2., 3., 4.]) In [2]: dU = np.array([0.1, 0.1, 0.1, 0.1]) In [3]: idx = np.array([0, 1, 2, 0]) In [4]: U Out[4]: array([ 1., 2., 3., 4.]) In [5]: U[idx] Out[5]: array([ 1., 2., 3., 1.]) In [6]: U[idx] += dU In [7]: U Out[7]: array([ 1.1, 2.1, 3.1, 4. ]) Ideally U would end up as array([ 1.2, 2.1, 3.1, 4. ]) Neil
Mon, 21 Dec 2009 09:35:08 +0000, Neil wrote: [clip]
I'm also interested to see if there are any answers to this; I came across a similar problem recently. It would have been convenient to do something like U[idx] += dU, but this didn't work because there were repeated indices in idx. Here's a short example that shows the problem:
In [1]: U = np.array([1., 2., 3., 4.]) In [2]: dU = np.array([0.1, 0.1, 0.1, 0.1]) In [3]: idx = np.array([0, 1, 2, 0]) In [4]: U Out[4]: array([ 1., 2., 3., 4.]) In [5]: U[idx] Out[5]: array([ 1., 2., 3., 1.]) In [6]: U[idx] += dU In [7]: U Out[7]: array([ 1.1, 2.1, 3.1, 4. ])
Ideally U would end up as array([ 1.2, 2.1, 3.1, 4. ])
One solution could be to use bincount: d = np.bincount(idx, dU) U[:len(d)] += d Also, bincount works only with scalar weights, so this is not a fully vectorized solution. A dedicated function could be nice here. -- Pauli Virtanen
On Mon, Dec 21, 2009 at 6:35 AM, Pauli Virtanen <pav+sp@iki.fi> wrote:
Mon, 21 Dec 2009 09:35:08 +0000, Neil wrote: [clip]
I'm also interested to see if there are any answers to this; I came across a similar problem recently. It would have been convenient to do something like U[idx] += dU, but this didn't work because there were repeated indices in idx. Here's a short example that shows the problem:
In [1]: U = np.array([1., 2., 3., 4.]) In [2]: dU = np.array([0.1, 0.1, 0.1, 0.1]) In [3]: idx = np.array([0, 1, 2, 0]) In [4]: U Out[4]: array([ 1., 2., 3., 4.]) In [5]: U[idx] Out[5]: array([ 1., 2., 3., 1.]) In [6]: U[idx] += dU In [7]: U Out[7]: array([ 1.1, 2.1, 3.1, 4. ])
Ideally U would end up as array([ 1.2, 2.1, 3.1, 4. ])
One solution could be to use bincount:
d = np.bincount(idx, dU) U[:len(d)] += d
Also, bincount works only with scalar weights, so this is not a fully vectorized solution.
A dedicated function could be nice here.
Or what would be *very* useful in many applications, is to extend bincount to take nd weights. Josef
-- Pauli Virtanen
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
So with bincount I can exchange a loop over P for a loop over M? I guess for me that is still really helpful. Thanks! - George On Mon, Dec 21, 2009 at 6:35 AM, Pauli Virtanen <pav+sp@iki.fi> wrote:
Mon, 21 Dec 2009 09:35:08 +0000, Neil wrote: [clip]
I'm also interested to see if there are any answers to this; I came across a similar problem recently. It would have been convenient to do something like U[idx] += dU, but this didn't work because there were repeated indices in idx. Here's a short example that shows the problem:
In [1]: U = np.array([1., 2., 3., 4.]) In [2]: dU = np.array([0.1, 0.1, 0.1, 0.1]) In [3]: idx = np.array([0, 1, 2, 0]) In [4]: U Out[4]: array([ 1., 2., 3., 4.]) In [5]: U[idx] Out[5]: array([ 1., 2., 3., 1.]) In [6]: U[idx] += dU In [7]: U Out[7]: array([ 1.1, 2.1, 3.1, 4. ])
Ideally U would end up as array([ 1.2, 2.1, 3.1, 4. ])
One solution could be to use bincount:
d = np.bincount(idx, dU) U[:len(d)] += d
Also, bincount works only with scalar weights, so this is not a fully vectorized solution.
A dedicated function could be nice here.
-- Pauli Virtanen
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
participants (4)
-
George Dahl -
josef.pktd＠gmail.com -
Neil -
Pauli Virtanen