[Matrix-SIG] Limited size arrays with cPickle

Herbert L. Roitblat roitblat@hawaii.edu
Sat, 1 Jan 2000 18:55:58 -1000

Yes, I did.  I have made that mistake in the past.  It does work with
smaller arrays, but this is the biggest one I have tried to do.

Thank you for asking.
-----Original Message-----
From: David Ascher <da@ski.org>
To: Herbert L. Roitblat <roitblat@hawaii.edu>; matrix-SIG@python.org
Date: Saturday, January 01, 2000 5:50 PM
Subject: Re: [Matrix-SIG] Limited size arrays with cPickle

>Did you make sure to use "binary" pickle to a file opened in 'wb' mode?
>----- Original Message -----
>From: Herbert L. Roitblat <roitblat@hawaii.edu>
>To: <matrix-SIG@python.org>
>Sent: Saturday, January 01, 2000 4:41 PM
>Subject: [Matrix-SIG] Limited size arrays with cPickle
>> Help Please:
>> I have run into this problem before and I have a workaround for it now,
>> it seems like something that would be good to fix.
>> I have a very large array (19741 x 53566) stored as 8-bit unsigned.  I
>> to cPickle it into a file and then cPickle it back when I want to use it.
>> have a 2 GB machine, so memory is not immediately the problem, rather it
>> seems to be the routine that translates the array into a string for
>> It reports a memory error.  I suppose that it could take up that much
>> memory to translate this array into a string, but it seems that there
>> to be a more efficient way to store arrays that would not take any more
>> memory than the amount taken by the array.  The actual routine that does
>> translation is in multiarray.pyd, so I cannot get to it with my limited
>> skills.
>> I'm running NT on a Pentium II 450MHz machine.  My Numeric is dated July
>> 1999, but I think that this is the date I downloaded it.  I believe that
>> is the latest release of Numeric.  The pdf manual is copyright 1999.
>> Thank you.
>> Herbert L. Roitblat, Ph.D.
>> Department of Psychology
>> University of Hawaii
>> 2430 Campus Road
>> Honolulu, HI 96822
>> _______________________________________________
>> Matrix-SIG maillist  -  Matrix-SIG@python.org
>> http://www.python.org/mailman/listinfo/matrix-sig
>Matrix-SIG maillist  -  Matrix-SIG@python.org