[Numpy-discussion] MemoryError in numpy.test() on AIXinstallation

[Charles R Harris]

On Mon, Apr 27, 2009 at 1:59 PM, Whitcomb, Mr. Tim <
tim.whitcomb at nrlmry.navy.mil> wrote:

> >> ----------------------------------
> >>              import numpy as np
> >>              from cStringIO import StringIO
> >>              from numpy.lib import format
> >>
> >>              f = StringIO()
> >>              a = np.array([], dtype='object')
> >>              format.write_array(f, a)
> >>              f2 = StringIO(f.getvalue())
> >>              a2 = format.read_array(f2)
> >>
> >>              print('%s %s' % (a, a2))
> >>              -----------------------------------
>
> >>              On the AIX/PPC install, this crashes with:
> >>              Traceback (most recent call last):
> >>               File "<stdin>", line 1, in <module>
> >>               File "crash_numpy.py", line 10, in <module>
> >>                 a2  = format.read_array(f2)
> >>               File
> >>
> "$HOME/.local/python/lib/python2.5/site-packages/numpy/lib/format.py",
> >>              line 362, in read_array
> >>                 array = cPickle.load(fp)
> >>              MemoryError
>
> >       I think this means out of memory, which is probably not correct.
> What happens if you simply pickle and object array? Something like
> >
> >       If that fails, try it using pickle instead of cPickle, then with
> a non-empty object array like a = np.array([None,None], dtype='object')
>
> Just trying to pickle an empty object array:
> -------
> pickle_file = 'foo.pkl'
> a = np.array([], dtype='object')
> cPickle.dump(a, open(pickle_file, 'w'))
> b = cPickle.load(open(pickle_file, 'r'))
> print(b)
> -------
> Traceback (most recent call last):
>   File "c_harris_test.py", line 7, in <module>
>    b = cPickle.load(open(pickle_file, 'r'))
> MemoryError
>
> Changing 'cPickle' to 'pickle' crashes again, with:
> Traceback (most recent call last):
>   File "c_harris_test.py", line 7, in <module>
>    b = pickle.load(open(pickle_file, 'r'))
>  File "/site/python-2.5.2/lib/python2.5/pickle.py", line 1370, in load
>    return Unpickler(file).load()
>  File "/site/python-2.5.2/lib/python2.5/pickle.py", line 858, in load
>    dispatch[key](self)
>  File "/site/python-2.5.2/lib/python2.5/pickle.py", line 1217, in
> load_build
>    setstate(state)
> MemoryError
>
> If I try either pickle or cPickle with a non-empty object (e.g.
> np.array([None])), it works fine.  Using np.array([ [] ]) crashes as
> well.  If I take Numpy out of the loop, I can successfully pickle an
> empty list with no issues.  Would platform endian-ness cause a problem?
>

It could, but I think AIX on the PPC is big endian, as is OSX, and I would
expect endian problems to show up in many other places, not just for empty
arrays. Hmm...

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20090427/32cdc247/attachment.html>