Help with finding how much memory a variable is using
rich_somerfield at tertio.com
Mon Feb 26 10:11:59 CET 2001
<s713221 at student.gu.edu.au> wrote in message
news:3A986565.BAF4E05B at student.gu.edu.au...
> Rich Somerfield wrote:
> > Hi,
> > I am generating a huge, huge list of data (cant think of a decent way to
> > reduce the required storage size and have it in a usable form).
> > try to keep this list for future derivations I get a memory problem from
> > Windows (effectively terminating my python script).
> > I presume this is because of the huge list. Is it possible to find out
> > amount of memory a variable [not a type of the variable, the actual data
> > contained within the variable] is taking up? This variable is local to
> > class and it would appear that it is when i go out of scope of that
> > that the memory error is being issued. Would I be better making this
> > variable global (across all classes)? e.g. would this problem still
> > if the variable was not just local to 1 class?
> > Would a different python datatype solve this problem for me ?
> > If I cant handle this amount of data then I will have to think of a way
> > better representing it, this will be unfortunate though as and
> > reduction of data will [as far as i can think] vastly reduce
> > which is already a concern.
> > Regards
> > Rich
> Do you have to load all the list into memory, or is there a way you can
> load in parts and work on that sequentially. I haven't had a chance to
> use persistance modules yet, but their documentation may be a place to
> Joal Heagney/AncientHart
Unfortunately I need all of the data in memory at the same time. The only
way I can see to solve this problem is to reduce the memory consumption of
my application. And the only way I can see how to do that is by vastly
reducing the performance.
Do u know of any information that suggests how large a datatype can be in
Python (are there any upper limits?) ?
More information about the Python-list