Python 2.6 still not giving memory back to the OS...
"Martin v. Löwis"
martin at v.loewis.de
Sun Aug 16 19:22:52 CEST 2009
> As far as releasing memory back to the OS is concerned, I have dim
> memories of *x systems where free() would return space to the OS if
> the block was "large" and it was next to the "break" point ... this
> effect could be what you are seeing.
Today, there are two cases when malloc returns memory on a typical
Unix system (in particular, in Linux malloc):
a) if the malloc block block is small (below page size), it is allocated
from the brk heap, where it can only be returned if the last page of
that heap is completely free, and
b) if the block is large (multiple pages), it gets returned to the
system right away.
Case b) can only happen if the C malloc uses mmap to allocate large
blocks. For Python, case b) is realistic, in the sense that most
allocations go to Python's obmalloc, and those allocate from C malloc
in chunks of 256kiB (IIRC). So when such an arena is completely free
(not a single Python object allocated on it anymore), it can get
returned to the system.
In some case, Python also allocates smaller blocks from C malloc; in
this case, a) will trigger. So as long as something at the end of the
heap stays allocated, C malloc will not return anything from the brk
heap to the system.
More information about the Python-list