[Numpy-discussion] dimensions too large error

[David Cournapeau]

On Fri, 2008-03-14 at 18:00 -0700, Dinesh B Vadhia wrote:
> For the following code:
>  
> I = 18000
> J = 33000
> filename = 'ij.txt'
> A = scipy.asmatrix(numpy.empty((I,J), dtype=numpy.int))

You are asking to create a matrix of more than 2 Gb, which is unlikely
to work on a 32 bits OS (by default, most OS I know limit the memory
available to one process to 2 Gb). Even if you could go beyond this
limit (say 3 Gb of virtual adress space per process), you would still
certainly have problems because 2Gb of contiguous adress space is really
big.

So either you have to use smaller matrices, or to use a 64 bits OS.

cheers,

David