[Python-Dev] The untuned tunable parameter ARENA_SIZE
Victor Stinner
victor.stinner at gmail.com
Thu Jun 1 04:37:01 EDT 2017
2017-06-01 10:23 GMT+02:00 INADA Naoki <songofacandy at gmail.com>:
> AFAIK, allocating arena doesn't eat real (physical) memory.
>
> * On Windows, VirtualAlloc is used for arena. Real memory page is assigned
> when the page is used first time.
> * On Linux and some other *nix, anonymous mmap is used. Real page is
> assigned when first touch, like Windows.
Memory fragmentation is also a real problem in pymalloc. I don't think
that pymalloc is designed to reduce the memory fragmentation.
I know one worst case: the Python parser which allocates small objects
which will be freed when the parser completes, while other objects
living longer are created.
https://github.com/haypo/misc/blob/master/memory/python_memleak.py
In a perfect world, the parser should use a different memory allocator
for that. But currently, the Python API doesn't offer this level of
granularity.
Victor
More information about the Python-Dev
mailing list