Reduce memory fragmentation with MALLOC_MMAP_THRESHOLD_ and MALLOC_MMAP_MAX_
lbolla at gmail.com
Fri Feb 26 11:43:28 EST 2016
I've been experimenting with MALLOC_MMAP_THRESHOLD_ and MALLOC_MMAP_MAX_ env variables to affect memory management in a long-running Python 2 process.
I got the idea from this bug report: http://bugs.python.org/issue11849
The results I have are encouraging: memory fragmentation is reduced and the typical high-water mark visible in memory used by long-running processes is lower.
My only concern is if there are other side effects that may bite back, when using such low level tweaks. Does anyone have any experience in using them?
Here is an example script that shows how those variables affect RSS memory in a script that generate a large dictionary:
Just run "alloc.sh" and compare the output.
More information about the Python-list