[Python-Dev] The untuned tunable parameter ARENA_SIZE
victor.stinner at gmail.com
Thu Jun 1 05:03:04 EDT 2017
2017-06-01 10:41 GMT+02:00 Larry Hastings <larry at hastings.org>:
> On 06/01/2017 01:19 AM, Antoine Pitrou wrote:
> If you'd like to go that way anyway, I would suggest 1MB as a starting
> point in 3.7.
> I understand the desire for caution. But I was hoping maybe we could
> experiment with 4mb in trunk for a while? We could change it to 1mb--or
> even 256k--before beta 1 if we get anxious.
While I fail to explain why in depth, I would prefer to *not* touch
the default arena size at this point.
We need more data, for example measure the memory usage on different
workloads using different arena sizes.
It's really hard to tune a memory allocator for *any* use cases.
A simple enhancement would be to add an environment variable to change
the arena size at Python startup. Example: PYTHONARENASIZE=1M. If you
*know* that your application will allocate at least 2 GB, you may even
want to try PYTHONARENASIZE=1G which is more likely to use a single
large page... Such parameter cannot be used by default: it would make
the default Python memory usage insane ;-)
More information about the Python-Dev