[Tutor] Question about the memory manager

[eryk sun]

On Thu, Jan 14, 2016 at 3:03 AM, Albert-Jan Roskam
<sjeik_appie at hotmail.com> wrote:
>
> These two pages are quite nice. The author says the memory used by small objects is
> never returned to the OS, which may be problematic for long running processes.

The article by Evan Jones discusses a patch to enable releasing unused
arenas (i.e. "how the problem was fixed"). Starting with 2.5, unused
arenas do get released back to the heap. Here's the diff in which Tim
Peters merged in a "heavily altered derivative" of Evan's patch [1].

Also, 2.7 and 3.3 bypass C malloc/free and the process heap to instead
use mmap/munmap on POSIX when available. This avoids the heap
high-water mark problem. Similarly, 3.4 switched to using
VirtualAlloc/VirtualFree on Windows. 3.4 also introduced the
PyObjectArenaAllocator and associated C API functions [2] to allow
modifying the default allocators.

[1]: https://hg.python.org/cpython/diff/685849bd905c/Objects/obmalloc.c
[2]: https://docs.python.org/3/c-api/memory.html#customize-pyobject-arena-allocator