[Numpy-discussion] advanced indexing bug with huge arrays?
sturla at molden.no
Tue Jan 24 05:01:53 EST 2012
On 24.01.2012 10:15, Robert Kern wrote:
> There are two different uses of long that you need to distinguish. One
> is for sizes, and one is for parameters and values. The sizes should
> definitely be upgraded to npy_intp. The latter shouldn't; these should
> remain as the default integer type of Python and numpy, a C long.
Ok, that makes sence.
> The reason longs are used for sizes is that I wrote mtrand for Numeric
> and Python 2.4 before numpy was even announced (and I don't think we
> had npy_intp at the time I merged it into numpy, but I could be
> wrong). Using longs for sizes was the order of the day. I don't think
> I had even touched a 64-bit machine that wasn't a DEC Alpha at the
> time, so I knew very little about the issues.
On amd64 the "native" datatypes are actually a 64 bit pointer with a 32
bit offset (contrary to what we see in Python and NumPy C sources),
which is one reason why C longs are still 32 bits in MSVC. Thus an array
size (size_t) should be 64 bits, but array indices (C long) should be 32
bits. But nobody likes to code like that (e.g. we would need an extra 64
bit pointer as cursor if the buffer size overflows a C long), and I
don't think using a non-native 64-bit offset incur a lot of extra
overhead for the CPU.
More information about the NumPy-Discussion