[Python-Dev] Re: Activating pymalloc

[Tim Peters]

[Vladimir Marangozov]
> ...
> About tuning pymalloc: I would suggest reviving the profiler which
> sleeps in the patches list. You'll find it invaluable for gathering
> the stats directly from Python.

I believe it's invaluable for gathering statistics, but I'm not sure the
statistics themselves are invaluable anymore.  The detailed statistics on

    http://starship.python.net/crew/vlad/pymalloc/

were interesting at the time, and helped you decide on the *form* of the
Python allocator, but once that was decided they're not really compelling
anymore.  For example, none of the programs profiled there use Python
classes heavily, so they're not going to show that pymalloc currently can
never handle an empty dict (which is over the current small object
threshold).  I don't need histograms to tell me that, and don't need more
than a stopwatch <wink> to tell me whether enabling pymalloc allows a large
class-heavy program to run faster if the threshold gets boosted.  If
enabling pymalloc *doesn't* help such a program, then the statistics get
interesting again.  Else they're just going to show me "oh ya, class-heavy
programs use a lot of dicts, and pymalloc always wastes time passing those
on to the system malloc now".

> After some profiling, you'll have good reasons to change a parameter
> or two (like the small block threshold).

I expect we already have good reason to boost that, because of dicts alone.
This is something to be disproved more than justified (IMO).

> Spacing the size classes by 4 bytes is not good (well, *was* not good
> at the time). Spacing them by 16 might be good now.  I don;t know --
> just try it.  Changing the pool size might be good as well.

I'm not keen to fiddle any of those.  That would be real work <wink>.

> But if the performace deltas you get are relatively small, this would
> be a platform-specific effect, so just leave things as they are.

Absolutely.