Python resident memory retention & Evan Jones' improvements

[Matt Ernst]

On Oct 2, 4:47 pm, Andrew MacIntyre <andy... at bullseye.apana.org.au>
wrote:
> There are two things you need to be aware of in this situation:
>
> - not all Python's memory is allocated through Python's specialised
>    malloc() - int and float objects in particular (in 2.x at least) are
>    allocated directly via a privately managed free list, and any
>    allocation requiring more than 256 bytes is directed to the platform
>    malloc().  Any memory not allocated via Python's malloc() is not
>    subject to the memory release facility referred to above.
>
>    Python 2.6 does improve the management of memory consumed by int and
>    float objects via the garbage collector.
>
> - while Python attempts to maximally utilise memory arenas to improve
>    the chances of being able to free them, Python's malloc() does not do
>    any compaction of memory (ie moving allocations between arenas) within
>    existing arenas.  Nor does garbage collection do this.  So fragmented
>    allocations can cause the retention of nearly empty arenas.
>
> I suspect that what you see with the second test script above is caused
> by some sort of fragmentation.

Thank you for the explanation. Since my real application uses many
small objects, it makes sense that memory cannot be reclaimed in the
absence of compaction.

Matt