[Numpy-discussion] NumPy on 64-bit Linux (Opteron)

[Sebastian Haase]

(RE: numarray)
Hi, 
On an Opteron (Suse9) I was happy to see that I could allocate the memory for 
an Int32-1024x1024x1024 cube. (On a P4 a get 'MemoryError').
I remember that Todd always said that python itself wasn't 64bit-ready yet.
But why is it needed to support >4GB python-lists if one is only interested in 
large numarrays (or numeric) ?

Just for sake of completeness here is one traceback I got:
>>> import numarray
>>> a = numarray.array(shape=(1024,1024,1024), type=numarray.Int)
>>> a.type()
Int32
>>> a.shape
(1024, 1024, 1024)
>>> len(a)
1024
>>> a
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/home/haase/myCVS/numarray/build/lib.linux-x86_64-2.3/numarray/
numarraycore.py", line 896, in __repr__
    return array_repr(self)
  File "/home/haase/myCVS/numarray/build/lib.linux-x86_64-2.3/numarray/
numarraycore.py", line 1542, in array_repr
    lst = arrayprint.array2string(
  File "/home/haase/myCVS/numarray/build/lib.linux-x86_64-2.3/numarray/
arrayprint.py", line 188, in array2string
    separator, prefix)
  File "/home/haase/myCVS/numarray/build/lib.linux-x86_64-2.3/numarray/
arrayprint.py", line 137, in _array2string
    data = _leading_trailing(a)
  File "/home/haase/myCVS/numarray/build/lib.linux-x86_64-2.3/numarray/
arrayprint.py", line 111, in _leading_trailing
    l = [_leading_trailing(a[i]) for i in range(
  File "/home/haase/myCVS/numarray/build/lib.linux-x86_64-2.3/numarray/
arrayprint.py", line 111, in _leading_trailing
    l = [_leading_trailing(a[i]) for i in range(
  File "/home/haase/myCVS/numarray/build/lib.linux-x86_64-2.3/numarray/
arrayprint.py", line 105, in _leading_trailing
    b = _gen.concatenate((a[:_summaryEdgeItems],
  File "/home/haase/myCVS/numarray/build/lib.linux-x86_64-2.3/numarray/
generic.py", line 1061, in concatenate
    return _concat(arrs)
  File "/home/haase/myCVS/numarray/build/lib.linux-x86_64-2.3/numarray/
generic.py", line 1051, in _concat
    dest[ix:ix+_shape0(a)]._copyFrom(a)
numarray.libnumarray.error: copy4bytes: access beyond buffer. offset=11 
buffersize=0

Thanks 
Sebastian Haase




On Tuesday 08 February 2005 08:44 am, David M. Cooke wrote:
> On Tue, Feb 08, 2005 at 12:44:18PM +0100, konrad.hinsen at laposte.net wrote:
> > Is anyone here using NumPy on Opteron machines running Linux in 64 bit
> > mode? I am running into problems such as:
> >
> > Python 2.4 (#4, Jan 18 2005, 18:06:45)
> > [GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-42)] on linux2
> > Type "help", "copyright", "credits" or "license" for more information.
> >
> > >>> from Numeric import *
> > >>> a = arange(4)
> > >>> a
> >
> > array([0, 1, 2, 3])
> >
> > >>> a.shape = (2, 2)
> >
> > Traceback (most recent call last):
> >   File "<stdin>", line 1, in ?
> > ValueError: total size of new array must be unchanged
> >
> > This is with Python 2.4 and Numeric 23.7. Before looking into this
> > myself, I thought I'd ask here, perhaps someone has already found a
> > fix.
>
> Yes; I submitted a patch (#1108739) to the Numeric patch tracker last week.
>
> Basically, PyArray_Reshape takes an array of dimensions. That has to be
> a PyArray_INT (C ints) instead of the PyArray_LONG (C longs) that it has
> in the code, because that array is passed to
> PyArray_FromDimsAndDataAndDescr.