On Tue, May 14, 2013 at 11:26 AM, Toder, Evgeny <evgeny.toder@jpmorgan.com>wrote:
Hello,****
** **
One of the test cases in test_einsum causes integer overflow for i2 type. The test goes like this:****
** **
import numpy as np****
dtype = 'i2'****
n = 15****
a = np.arange(4*n, dtype=dtype).reshape(4,n)****
b = np.arange(n*6, dtype=dtype).reshape(n,6)****
c = np.arange(24, dtype=dtype).reshape(4,6)****
** **
It then calculates AxB using einsum. The problem is that the values in the last row of the result do not fit into i2:****
** **
np.einsum("ij,jk", a, b, dtype='f8', casting='unsafe')****
array([[ 6090., 6195., 6300., 6405., 6510., 6615.],****
[ 15540., 15870., 16200., 16530., 16860., 17190.],****
[ 24990., 25545., 26100., 26655., 27210., 27765.],****
[ 34440., 35220., 36000., 36780., 37560., 38340.]])****
** **
In my build this produces different results depending on whether out or .astype is used:****
** **
np.einsum("ij,jk", a, b, dtype='f8', casting='unsafe').astype(dtype)** **
array([[ 6090, 6195, 6300, 6405, 6510, 6615],****
[ 15540, 15870, 16200, 16530, 16860, 17190],****
[ 24990, 25545, 26100, 26655, 27210, 27765],****
[-31096, -30316, -29536, -28756, -27976, -27196]], dtype=int16)****
** **
np.einsum("ij,jk", a, b, out=c, dtype='f8', casting='unsafe')****
array([[ 6090, 6195, 6300, 6405, 6510, 6615],****
[ 15540, 15870, 16200, 16530, 16860, 17190],****
[ 24990, 25545, 26100, 26655, 27210, 27765],****
[-32768, -32768, -32768, -32768, -32768, -32768]], dtype=int16)****
** **
The test wants these (actually the same using numpy.dot) to be equal, so this difference causes it to fail. Both ways to handle overflow seem reasonable to me. ****
** **
Does numpy in general assign a defined behavior to integer overflow (e.g. two’s complement)?****
Is this use of integer overflow in the test intentional and is expected to work, or is my build broken?****
I don't know if it was intended in the test, but numpy doesn't guard against integer overflow. Rather, it takes the C approach and does modular arithmetic. I don't know how to fix that or even if it could be at this point given the historical lack of support. I suppose at some point there could be user type that made such checks, but it would be slower. Chuck