[Python-Dev] pymalloc killer

[Martin v. Loewis]

Tim Peters <tim.one@comcast.net> writes:

> So I have a variant of pymalloc that doesn't use magic cookies -- it
> knows "for sure", by maintaining a sorted (by address) contiguous
> vector of the base addresses of the (256KB each) "arenas" allocated
> by pymalloc.

I would not like to see such a binary search performed. Instead, if
anything needs to be done, I'd be in favour of using a constant-time
algorithm, even if it means that a littler more memory overhead is
necessary.

I have the following idea: each chunk allocated by ought to have a
pointer to its pool, immediately preceding the memory block. This will
make an overhead of 4 bytes per allocation. Chunks allocated by the
system allocator will have a null pointer preceding them.

To deal with alignment, the size classes would increase each by 4
bytes, so they would be spaced at 12, 20, 28, etc. bytes. With the 4
byte overallocation, each chunk would be 8-aligned, if the previous
chunk in the same pool is.

This approach would allow to remove the requirement that each pool
must be 4k-aligned.

To support the NULL pointer in a system-malloc'ed chunk, pymalloc
would overallocate 8 bytes if it defers to system malloc, to preserve
alignment.

What do you think?

Regards,
Martin