[issue21220] Enhance obmalloc allocation strategy

[Kristján Valur Jónsson]

Kristján Valur Jónsson added the comment:

Antoine:  The location of the arenas when they're individually allocated with mmap does not matter, no, but preferring to keep low address ones reduces vmem fragmentation, since they end up being clustered together in memory.  For the usable-arenas list, there is no extra O(n) because they were ordered anyway.  the effect of ARENA_STRATEGY is minor, but it helps for it to be consistent with POOL_STRATEGY.

The real win however is with POOL_STRATEGY.  Fragmentation is dramatically reduced.  This is demonstrated with the tools/scripts/memcrunch.py which you can use to experiment with it.  Performance e.g. of unittests also goes up.  The fact that there is a new O(n) sort operation when a pool becomes 'used' does not seem to matter for that.

Victor: I've tested using windows LFH many times before, the python obmalloc generally is much faster than that.  Annoying :).  It is actually a very good allocator.

The innovation here is the "lowest address strategy" which I have never seen before (it might be known, but then I'm not a CS) but is one that I have experimented with for often in the past.  It is suprisingly effective.  When there is memory churn, memory usage tends to migrate towards low addresses and free up memory.  Go ahead, try the scripts and see what happens.  The proof is in the pudding :)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue21220>
_______________________________________