[Python-Dev] The untuned tunable parameter ARENA_SIZE

Larry Hastings larry at hastings.org
Thu Jun 1 14:17:12 EDT 2017



On 06/01/2017 02:03 AM, Victor Stinner wrote:
> 2017-06-01 10:41 GMT+02:00 Larry Hastings <larry at hastings.org>:
>> On 06/01/2017 01:19 AM, Antoine Pitrou wrote:
>>> If you'd like to go that way anyway, I would suggest 1MB as a starting
>>> point in 3.7.
>>
>> I understand the desire for caution.  But I was hoping maybe we could
>> experiment with 4mb in trunk for a while?  We could change it to 1mb--or
>> even 256k--before beta 1 if we get anxious.
> While I fail to explain why in depth, I would prefer to *not* touch
> the default arena size at this point.
>
> We need more data, for example measure the memory usage on different
> workloads using different arena sizes.

I can't argue with collecting data at this point in the process.  My 
thesis is simply "the correct value for this tunable parameter in 2001 
is probably not the same value in 2017".  I don't mind proceeding 
*slowly* or gathering more data or what have you for now.  But I would 
like to see it change somehow between now and 3.7.0b1, because my sense 
is that we can get some performance for basically free by updating the 
value.

If ARENA_SIZE tracked Moore's Law, meaning that we doubled it every 18 
months like clockwork, it'd currently be 2**10 times bigger: 256MB, and 
we'd be changing it to 512MB at the end of August.

(And yes, as a high school student I was once bitten by a radioactive 
optimizer, so these days when I'm near possible optimizations my 
spider-sense--uh, I mean, my optimization-sense--starts tingling.)

> A simple enhancement would be to add an environment variable to change
> the arena size at Python startup. Example: PYTHONARENASIZE=1M.

Implementing this would slow down address_in_range which currently 
compiles in arena size.  It'd be by a tiny amount, but this inline 
function gets called very very frequently.  It's possible this wouldn't 
hurt performance, but my guess is it'd offset the gains we got from 
larger arenas, and the net result would be no faster or slightly slower.


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20170601/2f3e36ea/attachment-0001.html>


More information about the Python-Dev mailing list