[Numpy-discussion] Integer overflow in test_einsum (1.7.1)

Charles R Harris charlesr.harris at gmail.com
Tue May 14 14:25:14 EDT 2013


On Tue, May 14, 2013 at 11:26 AM, Toder, Evgeny
<evgeny.toder at jpmorgan.com>wrote:

>  Hello,****
>
> ** **
>
> One of the test cases in test_einsum causes integer overflow for i2 type.
> The test goes like this:****
>
> ** **
>
> >>> import numpy as np****
>
> >>> dtype = 'i2'****
>
> >>> n = 15****
>
> >>> a = np.arange(4*n, dtype=dtype).reshape(4,n)****
>
> >>> b = np.arange(n*6, dtype=dtype).reshape(n,6)****
>
> >>> c = np.arange(24, dtype=dtype).reshape(4,6)****
>
> ** **
>
> It then calculates AxB using einsum. The problem is that the values in the
> last row of the result do not fit into i2:****
>
> ** **
>
> >>> np.einsum("ij,jk", a, b, dtype='f8', casting='unsafe')****
>
> array([[  6090.,   6195.,   6300.,   6405.,   6510.,   6615.],****
>
>        [ 15540.,  15870.,  16200.,  16530.,  16860.,  17190.],****
>
>        [ 24990.,  25545.,  26100.,  26655.,  27210.,  27765.],****
>
>        [ 34440.,  35220.,  36000.,  36780.,  37560.,  38340.]])****
>
> ** **
>
> In my build this produces different results depending on whether out or
> .astype is used:****
>
> ** **
>
> >>> np.einsum("ij,jk", a, b, dtype='f8', casting='unsafe').astype(dtype)**
> **
>
> array([[  6090,   6195,   6300,   6405,   6510,   6615],****
>
>        [ 15540,  15870,  16200,  16530,  16860,  17190],****
>
>        [ 24990,  25545,  26100,  26655,  27210,  27765],****
>
>        [-31096, -30316, -29536, -28756, -27976, -27196]], dtype=int16)****
>
> ** **
>
> >>> np.einsum("ij,jk", a, b, out=c, dtype='f8', casting='unsafe')****
>
> array([[  6090,   6195,   6300,   6405,   6510,   6615],****
>
>        [ 15540,  15870,  16200,  16530,  16860,  17190],****
>
>        [ 24990,  25545,  26100,  26655,  27210,  27765],****
>
>        [-32768, -32768, -32768, -32768, -32768, -32768]], dtype=int16)****
>
> ** **
>
> The test wants these (actually the same using numpy.dot) to be equal, so
> this difference causes it to fail. Both ways to handle overflow seem
> reasonable to me. ****
>
> ** **
>
> Does numpy in general assign a defined behavior to integer overflow (e.g.
> two’s complement)?****
>
> Is this use of integer overflow in the test intentional and is expected to
> work, or is my build broken?****
>

I don't know if it was intended in the test, but numpy doesn't guard
against integer overflow. Rather, it takes the C approach and does modular
arithmetic. I don't know how to fix that or even if it could be at this
point given the historical lack of support. I suppose at some point there
could be user type that made such checks, but it would be slower.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130514/bf2c45dc/attachment.html>


More information about the NumPy-Discussion mailing list