On Fri, Jan 4, 2013 at 12:39 AM, Peter Cock
Since I've actually used NumPy arrays with specific low memory types, I thought I should comment about my use case if case it is helpful:
I've only used the low precision types like np.uint8 (unsigned) where I needed to limit my memory usage. In this case, the topology of a graph allowing multiple edges held as an integer adjacency matrix, A. I would calculate things like A^n for paths of length n, and also make changes to A directly (e.g. adding edges). So an overflow was always possible, and neither the old behaviour (type preserving but wrapping on overflow giving data corruption) nor the current behaviour (type promotion overriding my deliberate memory management) are nice. My preferences here would be for an exception, so I knew right away.
The other use case which comes to mind is dealing with low level libraries and/or file formats, and here automagic type promotion would probably be unwelcome.
Regards,
Peter
Elsewhere on the thread, Nathaniel Smith
To be clear: we're only talking here about the case where you have a mix of a narrow dtype in an array and a scalar value that cannot be represented in that narrow dtype. If both sides are arrays then we continue to upcast as normal. So my impression is that this means very little in practical terms, because this is a rare and historically poorly supported situation.
But if this is something you're running into in practice then you may have a better idea than us about the practical effects. Do you have any examples where this has come up that you can share?
-n
Clarification appreciated - on closer inspection for my adjacency matrix example I would not fall over the issue in https://github.com/numpy/numpy/issues/2878
import numpy as np np.__version__ '1.6.1' A = np.zeros((100,100), np.uint8) # Matrix could be very big A[3,4] = 255 # Max value, setting up next step in example A[3,4] 255 A[3,4] += 1 # Silently overflows on NumPy 1.6 A[3,4] 0
To trigger the contentious behaviour I'd have to do something like this:
A = np.zeros((100,100), np.uint8) B = A + 256 B array([[256, 256, 256, ..., 256, 256, 256], [256, 256, 256, ..., 256, 256, 256], [256, 256, 256, ..., 256, 256, 256], ..., [256, 256, 256, ..., 256, 256, 256], [256, 256, 256, ..., 256, 256, 256], [256, 256, 256, ..., 256, 256, 256]], dtype=uint16)
I wasn't doing anything like that in my code though, just simple matrix multiplication and in situ element modification, for example A[i,j] += 1 to add an edge. I still agree that for https://github.com/numpy/numpy/issues/2878 an exception sounds sensible. Peter