[Numpy-discussion] Numarray problem on Opteron system, Suse 9.1

[Jason Ruiter]

Greetings,

I'm running Suse 9.1 on a dual opteron system with 16GB of RAM.  I'm
using Python 2.3.3, Numeric 23.1 and numarray 0.9.

I'm trying to allocate large (>4GB) arrays.  Under Numeric, I get:

Python 2.3.3 (#1, Apr  6 2004, 09:45:08)
[GCC 3.3.3 (SuSE Linux)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from Numeric import *
>>> a=ones(2000000000,Int16);
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib64/python2.3/site-packages/Numeric/Numeric.py", line
578, in ones
    a=zeros(shape, typecode, savespace)
MemoryError: can't allocate memory for array
>>>

Under numarray:

Python 2.3.3 (#1, Apr  6 2004, 09:45:08)
[GCC 3.3.3 (SuSE Linux)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from numarray import *
>>> a=ones(2000000000,Int16)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib64/python2.3/site-packages/numarray/numarraycore.py",
line 1111, in ones
    retarr = _fillarray(_gen.product(shape), 1, 0, type)
  File "/usr/lib64/python2.3/site-packages/numarray/numarraycore.py",
line 144, in _fillarray
    outarr = NumArray((size,), outtype)
ValueError: new_memory: invalid region size: -294967296.
>>>


I've verified that python, Numeric, and numarray are built and linked
against the 64bit libraries.

I've also verified that it's possible to allocate a >16GB Array with the
following program I found on one of the debian mailling lists.
:q:q:q!
#include <stdio.h>
#include <stdlib.h>
int main()  {  size_t n;  void *p;  double gb;
  for(gb=20;gb>.3;gb-=.5) {
        n= 1024L * 1024L * 1024L * gb;
        p = malloc( n );
        printf("%12lu %4.1lfGb %p\n",n,n/1024./1024./1024.,p);
        free(p); }  return 0;  }

 21474836480 20.0Gb (nil)
 20937965568 19.5Gb (nil)
 20401094656 19.0Gb (nil)
 19864223744 18.5Gb (nil)
 19327352832 18.0Gb (nil)
 18790481920 17.5Gb (nil)
 18253611008 17.0Gb (nil)
 17716740096 16.5Gb 0x3995a02010
 17179869184 16.0Gb 0x3995a02010
 16642998272 15.5Gb 0x3995a02010
 16106127360 15.0Gb 0x3995a02010
 15569256448 14.5Gb 0x3995a02010
 15032385536 14.0Gb 0x3995a02010
 14495514624 13.5Gb 0x3995a02010
 13958643712 13.0Gb 0x3995a02010
 13421772800 12.5Gb 0x3995a02010
 12884901888 12.0Gb 0x3995a02010
 12348030976 11.5Gb 0x3995a02010
 11811160064 11.0Gb 0x3995a02010
 11274289152 10.5Gb 0x3995a02010
 10737418240 10.0Gb 0x3995a02010
 10200547328  9.5Gb 0x3995a02010
  9663676416  9.0Gb 0x3995a02010
  9126805504  8.5Gb 0x3995a02010
  8589934592  8.0Gb 0x3995a02010
  8053063680  7.5Gb 0x3995a02010
  7516192768  7.0Gb 0x3995a02010
  6979321856  6.5Gb 0x3995a02010
  6442450944  6.0Gb 0x3995a02010
  5905580032  5.5Gb 0x3995a02010
  5368709120  5.0Gb 0x3995a02010
  4831838208  4.5Gb 0x3995a02010
  4294967296  4.0Gb 0x3995a02010
  3758096384  3.5Gb 0x3995a02010
  3221225472  3.0Gb 0x3995a02010
  2684354560  2.5Gb 0x3995a02010
  2147483648  2.0Gb 0x3995a02010
  1610612736  1.5Gb 0x3995a02010
  1073741824  1.0Gb 0x3995a02010
   536870912  0.5Gb 0x3995a02010


Can someone point me in the right direction?

Thanks
Jason



-- 
Jason Ruiter
Research Scientist
Environment and Emerging Technologies Division
Altarum Institute
Ann Arbor, Michigan, USA
(v)734.302.4724 (f)734.302.4991 (m)248.345.4598
Jason.Ruiter at Altarum.org