Help with finding how much memory a variable is using

[s713221 at student.gu.edu.au]

Rich Somerfield wrote:
> 
> Hi,
> 
> I am generating a huge, huge list of data (cant think of a decent way to
> reduce the required storage size and have it in a usable form).  Everytime i
> try to keep this list for future derivations I get a memory problem from
> Windows (effectively terminating my python script).
> 
> I presume this is because of the huge list.  Is it possible to find out the
> amount of memory a variable [not a type of the variable, the actual data
> contained within the variable] is taking up?  This variable is local to a
> class and it would appear that it is when i go out of scope of that class
> that the memory error is being issued.  Would I be better making this
> variable global (across all classes)? e.g. would this problem still happen
> if the variable was not just local to 1 class?
> 
> Would a different python datatype solve this problem for me ?
> 
> If I cant handle this amount of data then I will have to think of a way of
> better representing it, this will be unfortunate though as and compression /
> reduction of data will [as far as i can think] vastly reduce performance -
> which is already a concern.
> 
> Regards
> Rich

Do you have to load all the list into memory, or is there a way you can
load in parts and work on that sequentially. I haven't had a chance to
use persistance modules yet, but their documentation may be a place to
start.

Joal Heagney/AncientHart