[Python-Dev] The untuned tunable parameter ARENA_SIZE

Larry Hastings larry at hastings.org
Thu Jun 1 23:21:12 EDT 2017


On 06/01/2017 02:50 AM, Antoine Pitrou wrote:
> Another possible strategy is: allocate several arenas at once (using a
> larger mmap() call), and use MADV_DONTNEED to relinquish individual
> arenas.

Thus adding a *fourth* layer of abstraction over memory we get from the OS?

    block -> pool -> arena -> "multi-arena" -> OS

Y'know, this might actually make things faster.  These multi-arenas 
could be the dynamically growing thing Victor wants to try.  We allocate 
16mb, then carve it up into arenas (however big those are), then next 
time allocate 32mb or what have you. Since the arenas remain a fixed 
size, we don't make the frequently-used code path (address_in_range) any 
slower.  The code to deal with the multi-arenas would add a little 
complexity--to an admittedly already complex allocator implementation, 
but then what allocator isn't complex internally?--but it'd be an 
infrequent code path and I bet it'd be an improvement over simply 
calling malloc / mmap / VirtualAlloc.  What do you think, Victor?

And to think I started this reply ironically,


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20170601/46762760/attachment.html>


More information about the Python-Dev mailing list