[Matrix-SIG] Limited size arrays with cPickle

Herbert L. Roitblat roitblat@hawaii.edu
Sat, 1 Jan 2000 14:41:36 -1000


Help Please:

I have run into this problem before and I have a workaround for it now, but
it seems like something that would be good to fix.
I have a very large array (19741 x 53566) stored as 8-bit unsigned.  I want
to cPickle it into a file and then cPickle it back when I want to use it.  I
have a 2 GB machine, so memory is not immediately the problem, rather it
seems to be the routine that translates the array into a string for cPickle.
It reports a memory error.  I suppose that it could take up that much more
memory to translate this array into a string, but it seems that there ought
to be a more efficient way to store arrays that would not take any more
memory than the amount taken by the array.  The actual routine that does the
translation is in multiarray.pyd, so I cannot get to it with my limited
skills.

I'm running NT on a Pentium II 450MHz machine.  My Numeric is dated July 13,
1999, but I think that this is the date I downloaded it.  I believe that it
is the latest release of Numeric.  The pdf manual is copyright 1999.

Thank you.
Herbert L. Roitblat, Ph.D.
Department of Psychology
University of Hawaii
2430 Campus Road
Honolulu, HI 96822