[Numpy-discussion] advanced indexing bug with huge arrays?

[Robin]

On Tue, Jan 24, 2012 at 6:24 PM, David Warde-Farley
<wardefar at iro.umontreal.ca> wrote:
> On Tue, Jan 24, 2012 at 06:00:05AM +0100, Sturla Molden wrote:
>> Den 23.01.2012 22:08, skrev Christoph Gohlke:
>> >
>> > Maybe this explains the win-amd64 behavior: There are a couple of places
>> > in mtrand where array indices and sizes are C long instead of npy_intp,
>> > for example in the randint function:
>> >
>> > <https://github.com/numpy/numpy/blob/master/numpy/random/mtrand/mtrand.pyx#L863>
>> >
>> >
>>
>> Both i and length could overflow here. It should overflow on allocation
>> of more than 2 GB.
>>
>> There is also a lot of C longs in the internal state (line 55-105), as
>> well as the other functions.
>>
>> Producing 2 GB of random ints twice fails:
>
> Sturla, since you seem to have access to Win64 machines, do you suppose you
> could try this code:
>
>>>> a = numpy.ones((1, 972))
>>>> b = numpy.zeros((4993210,), dtype=int)
>>>> c = a[b]
>
> and verify that there's a whole lot of 0s in the matrix, specifically,
>
>>>> c[574519:].sum()
> 356.0
>>>> c[574520:].sum()
> 0.0
>
> is the case on Linux 64-bit; is it the case on Windows 64?

Yes - I get exactly the same numbers in 64 bit windows with 1.6.1.

Cheers

Robin