[Python-Dev] The untuned tunable parameter ARENA_SIZE
tim.peters at gmail.com
Fri Jun 2 15:09:42 EDT 2017
>> While I would like to increase the pool size, it's fraught with
[Antoine Pitrou <solipsis at pitrou.net>]
> What would be the point of increasing the pool size? Apart from being
> able to allocate 4KB objects out of it, I mean.
> Since 4KB+ objects are relatively uncommon (I mean we don't allocate
> hundreds of thousands of them per second), I don't think it's really
> worthwhile trying to have the small object allocator handle them.
I don't care about "large" objects here. It's educated intuition
about speed: so long as pymalloc is working within a pool, it's
blazing fast. When it has to muck with a new pool, much slower
(compared to staying within a pool) code is required. When it has to
muck with a new arena, slower still.
So the intuition is simple: the larger a pool, the more object
operations it can handle staying within its best-case (fastest) code
>> It would be nice to find a different way for pymalloc to figure out
>> which addresses belong to it. The excruciating Py_ADDRESS_IN_RANGE
>> manages to do it in small constant (independent of the number of
>> arenas and pools in use) time, which is its only virtue ;-)
> So, to sum it up, it's excruciating but fast and works reliably. Why
> change it?
To enable using pools larger than the greatest common divisor of all
OS's native pool sizes.
There's also that Py_ADDRESS_IN_RANGE is responsible for countless
hours of head-scratching by poor developers trying to use magical
memory debuggers - it's at best unusual for code to read up memory
without caring a bit whether the memory has ever been stored to.
I should note that Py_ADDRESS_IN_RANGE is my code - this isn't a
backhanded swipe at someone else. It's always been near the top of
the "code stink" scale. So I thought I'd mention it in case someone
has been sitting on a cleaner idea but has held back because they
didn't want to offend me ;-)
More information about the Python-Dev