python memory blow out

Tim Peters tim.peters at gmail.com
Wed Jan 26 20:59:11 EST 2005


[Simon Wittber]
>> According to the above post:
>>    a) If the allocation is > 256 bytes, call the system malloc.
>>    b) If the allocation is < 256, use its own malloc implementation, which
>>       allocates memory in 256 kB chunks and never releases it.
>>
>> I imagine this means that large memory allocations are eventually
>> released back to the operating system. However, in my case, this
>> appears to be not happening.

Large chunks are returned to the platform C when they become
unreferenced.  More below.

[Stephen Thorne]
> There was a recent patch posted to python-dev list which allows python
> to release memory back to the operating system once the 256kb chunk is
> no longer used.

Nope, the patch may return such a 256KB chunk to the platform C
library's free() function.  That's never what someone actually means
by the vague "released back to the operating system", and whether the
latter might happen is another platform-dependent (OS + C library +
specific versions of each) can of worms.

> I'm not saying running a bleeding edge CVS HEAD python plus untested
> development patches is going to be a solution for you in the short
> term, but I just wanted to mention it because I'm excited about this
> limitation disappearing in python :).

It's a necessary first step, anyway.  Alas, there's no way to take a
second step without messy platform-specific code (when does your
favorite platform C free() "release memory back" to your favorite
operating system?  to "do something about that", exact knowledge of
platform details is necessary).



More information about the Python-list mailing list