[Python-Dev] The untuned tunable parameter ARENA_SIZE

Larry Hastings larry at hastings.org
Thu Jun 1 04:07:02 EDT 2017

On 06/01/2017 12:57 AM, Victor Stinner wrote:
> I would prefer to have an adaptative arena size. For example start at
> 256 kB and then double the arena size while the memory usage grows.
> The problem is that pymalloc is currently designed for a fixed arena
> size. I have no idea how hard it would be to make the size per
> allocated arena.

It's not hard.  The major pain point is that it'd make the 
address_in_range() inline function slightly more expensive. Currently 
that code has ARENA_SIZE hardcoded inside it; if the size was dynamic 
we'd have to look up the size of the arena every time. This function is 
called every time we free a pointer, so it's done hundreds of thousands 
of times per second (as you point out).

It's worth trying the experiment to see if dynamic arena sizes would 
make programs notably faster.  However... why not both?  Changing to 4mb 
arenas now is a one-line change, and on first examination seems mostly 
harmless, and yields an easy (if tiny) performance win.  If someone 
wants to experiment with dynamic arenas, they could go right ahead, and 
if it works well we could merge that too.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20170601/aa09e9f3/attachment-0001.html>

More information about the Python-Dev mailing list