[issue26415] Fragmentation of the heap memory in the Python parser

[STINNER Victor]

STINNER Victor added the comment:

Benjamin Peterson: "It seems to me a simpler solution would be allocate all nodes for a parse tree in an arena."

Exactly, that's the real fix. Just make sure that deallocating this arena does punch holes in the "heap memory". For example, on Linux, it means using mmap() rather than sbrk() to allocate memory.


A. Skrobov: "An arena might help reclaim the memory once the parsing is complete, but it wouldn't reduce the peak memory consumption by the parser, and so it wouldn't prevent a MemoryError when parsing a 35MB source on a PC with 2GB of RAM."

Parsing a 35 MB source doesn't seem like a good idea :-) I think that it's ok to have a memory peak, but it's not ok to not release the memory later.

Do you have a solution avoid the memory peak *and* don't create memory fragmentation?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26415>
_______________________________________