[Python-Dev] The untuned tunable parameter ARENA_SIZE
larry at hastings.org
Thu Jun 1 14:17:12 EDT 2017
On 06/01/2017 02:03 AM, Victor Stinner wrote:
> 2017-06-01 10:41 GMT+02:00 Larry Hastings <larry at hastings.org>:
>> On 06/01/2017 01:19 AM, Antoine Pitrou wrote:
>>> If you'd like to go that way anyway, I would suggest 1MB as a starting
>>> point in 3.7.
>> I understand the desire for caution. But I was hoping maybe we could
>> experiment with 4mb in trunk for a while? We could change it to 1mb--or
>> even 256k--before beta 1 if we get anxious.
> While I fail to explain why in depth, I would prefer to *not* touch
> the default arena size at this point.
> We need more data, for example measure the memory usage on different
> workloads using different arena sizes.
I can't argue with collecting data at this point in the process. My
thesis is simply "the correct value for this tunable parameter in 2001
is probably not the same value in 2017". I don't mind proceeding
*slowly* or gathering more data or what have you for now. But I would
like to see it change somehow between now and 3.7.0b1, because my sense
is that we can get some performance for basically free by updating the
If ARENA_SIZE tracked Moore's Law, meaning that we doubled it every 18
months like clockwork, it'd currently be 2**10 times bigger: 256MB, and
we'd be changing it to 512MB at the end of August.
(And yes, as a high school student I was once bitten by a radioactive
optimizer, so these days when I'm near possible optimizations my
spider-sense--uh, I mean, my optimization-sense--starts tingling.)
> A simple enhancement would be to add an environment variable to change
> the arena size at Python startup. Example: PYTHONARENASIZE=1M.
Implementing this would slow down address_in_range which currently
compiles in arena size. It'd be by a tiny amount, but this inline
function gets called very very frequently. It's possible this wouldn't
hurt performance, but my guess is it'd offset the gains we got from
larger arenas, and the net result would be no faster or slightly slower.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Python-Dev