memory usage

[Alex Martelli]

Nagy Gabor wrote:

> On 03-May-08 07:35, Skip Montanaro wrote:
>>     Nagy> Can you access the list created by range later?
>> 
>> Sure.  Just assign it to a variable first:
>> 
>>     ilist = range(1, 600000)
>>     for i in ilist:
>>         ...
> 
> That's clear. But if I don't assign it to any variable, why does it still
> occupy space?

It doesn't (the list goes away when you exit the loop, if you just
build it on the fly with range) BUT the ints in it do (because the
int type caches a "free-list" of cells for future reuse).

>> It's complicated by the fact that even though Python deletes objects, the
>> underlying malloc() package may not release the memory back to the system
>> (for many reasons).
> 
> That's OK, as long as python can use the same memory for another object.

Memory ever used for an int can only be used for an int (same or different
value).  That's because the int type gets memory for 1000 or so instances
at a time and parcels it out as needed (to reduce per-object overhead due
to using so many malloc calls).


>> You can't force Python to give memory back to the system.  Any time an
>> object's reference count drops to zero Python deletes the object, but
>> that doesn't mean the memory will be returned to the operating system.
> 
> So probably what happens with range() above is that it allocates the
> memory for the list, then since it is not assigned to a variable, it
> destroys the list, the memory is not freed, but then a second range() will
> use the same space, thus not increasing the memory occupation?

Yes, basically.

> That is good enough for me. Sort of.

However, xrange can probably do better -- try that.

> What about those integers, floats and interned strings you have mentioned?
> If I read 'internable' from a file, assign it to a variable, then this
> variable goes out of scope, will this space be usable again?

Probably not (depends on the release &c).


> You wrote, these strings may become immortal on certain versions of
> python. Is mine (2.2.1) affected? How do these "immortal" interned strings
> behave? What is the purpose of interning them? Will two occurences share
> the same memory and the variables will just contain an "intern-pointer"?

Interning is basically for speed, but, sorry, it's too hard to predict
what happens unless you explicitly call the intern built-in.


Alex