High memory usage - program mistake or Python feature?

[Gerhard Häring]

Ben S wrote:
> Aahz wrote:
> 
>>The part about high-bit characters is a complete red herring.  Note
>>that you've got the file in memory twice: once from readlines() and
>>once from map().  Although the readlines() call only delivers a
>>temporary object, the memory consumed doesn't get returned to the OS.
> 
> Why not? Does Python keep it around in a pool, in case I ask for it
> again? And is this customizable at all?

Python depends on the platform's malloc implementation for that. And 
most of these are implemented so they do not return free memory to the 
kernel. Yes, the malloc implementation will reuse the free memory next 
time you request memory for a new object.

The reasons for this are that there are hardly any disavantages to this 
approach. If your app's memory usage really goes up to, say 500 MB, then 
goes down to, say 100 MB and stays there, the unused memory will 
eventually be paged out to disk (swap partition/swap file).

-- Gerhard