Array addition inconsistency
Dear all, After some surprise, I noticed an inconsistency while adding array slices:
a = np.arange(5) a[1:] = a[1:] + a[:-1] a array([0, 1, 3, 5, 7])
versus inplace
a = np.arange(5) a[1:] += a[:-1] a array([ 0, 1, 3, 6, 10])
My suspicition is that the second variant does not create intermediate storage, and thus works on the intermediate result, effectively performing a.cumsum(). This behaviour is certainly surprising, and leads to unexpected errors if used without testing. Best, Martin -- Dr. Martin Lüthi luethi@vaw.baug.ethz.ch VAW Glaciology http://www.vaw.ethz.ch/gz ETH Zürich http://people.ee.ethz.ch/~luethim 8093 Zürich Tel: +41 44 632 40 93
On Thu, Aug 29, 2013 at 12:00 PM, Martin Luethi
Dear all,
After some surprise, I noticed an inconsistency while adding array slices:
a = np.arange(5) a[1:] = a[1:] + a[:-1] a array([0, 1, 3, 5, 7])
versus inplace
a = np.arange(5) a[1:] += a[:-1] a array([ 0, 1, 3, 6, 10])
My suspicition is that the second variant does not create intermediate storage, and thus works on the intermediate result, effectively performing a.cumsum().
Correct. Not creating intermediate storage is the point of using augmented assignment. -- Robert Kern
On Thu, Aug 29, 2013 at 8:04 AM, Robert Kern
On Thu, Aug 29, 2013 at 12:00 PM, Martin Luethi
wrote: Dear all,
After some surprise, I noticed an inconsistency while adding array slices:
a = np.arange(5) a[1:] = a[1:] + a[:-1] a array([0, 1, 3, 5, 7])
versus inplace
a = np.arange(5) a[1:] += a[:-1] a array([ 0, 1, 3, 6, 10])
My suspicition is that the second variant does not create intermediate storage, and thus works on the intermediate result, effectively performing a.cumsum().
Correct. Not creating intermediate storage is the point of using augmented assignment.
This can be very sneaky.
a = np.arange(5) a[:-1] = a[:-1] + a[1:] a array([1, 3, 5, 7, 4])
a = np.arange(5) a[:-1] += a[1:] a array([1, 3, 5, 7, 4])
So, if someone is semi-careful and tries out that example, they might (incorrectly) assume that such operations are safe without realizing that it was safe because the values of a[1:] were ahead of the values of a[:-1] in memory. I could easily imagine a situation where views of an array are passed around only to finally end up in an in-place operation like this and sometimes be right and sometimes be wrong. Maybe there can be some simple check that could be performed to detect this sort of situation? Cheers! Ben Root
On Thu, Aug 29, 2013 at 9:39 AM, Benjamin Root
On Thu, Aug 29, 2013 at 8:04 AM, Robert Kern
wrote: On Thu, Aug 29, 2013 at 12:00 PM, Martin Luethi
wrote: Dear all,
After some surprise, I noticed an inconsistency while adding array slices:
a = np.arange(5) a[1:] = a[1:] + a[:-1] a array([0, 1, 3, 5, 7])
versus inplace
a = np.arange(5) a[1:] += a[:-1] a array([ 0, 1, 3, 6, 10])
My suspicition is that the second variant does not create intermediate storage, and thus works on the intermediate result, effectively performing a.cumsum().
Correct. Not creating intermediate storage is the point of using augmented assignment.
This can be very sneaky.
a = np.arange(5) a[:-1] = a[:-1] + a[1:] a array([1, 3, 5, 7, 4])
a = np.arange(5) a[:-1] += a[1:] a array([1, 3, 5, 7, 4])
So, if someone is semi-careful and tries out that example, they might (incorrectly) assume that such operations are safe without realizing that it was safe because the values of a[1:] were ahead of the values of a[:-1] in memory. I could easily imagine a situation where views of an array are passed around only to finally end up in an in-place operation like this and sometimes be right and sometimes be wrong. Maybe there can be some simple check that could be performed to detect this sort of situation?
I think the main message is that you don't use inplace operation with mutables unless you know what you are doing, and you really need them. inplace cumsum in python
a = range(5) for i in xrange(1, 5): a[i] += a[i-1] ... a [0, 1, 3, 6, 10]
Defensive programming. Josef
Cheers! Ben Root
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
participants (4)
-
Benjamin Root
-
josef.pktd@gmail.com
-
Martin Luethi
-
Robert Kern