[Python-Dev] Re: Activating pymalloc

[Tim Peters]

[Andrew MacIntyre]
> It works just fine when pymalloc is configured normally (~2% speedup).
> [when running]
> pystone.py, MAL's PyBench - nothing real world though...

Ah.  PyBench will be more interesting when the small object threshold is
boosted -- parts of it use oodles of tiny dicts, and pymalloc can't handle
them now.

[when redirecting all (I think) mallocs to go thru pymalloc]
> test_longexp was the pathological case.

It's a pathological case on some boxes without pymalloc.  IIRC, it takes so
long to finish on some flavor(s) of Mac that nobody has waited for it to
complete (we're talking over half an hour here).

I just tried it on my (Windows) box, after redefining PyMem_{MALLOC,
REALLOC, FREE} to resolve to _PyMalloc_{Malloc, Realloc, Free}.  Ahem.  That
was total disaster, since the call to PyMem_MALLOC at the end of
_PyMalloc_Malloc became unbounded recursion.  Doh!

I then further bashed obmalloc.c to call malloc/realloc/free directly.
test_longexp sped up significantly then, taking about 3 seconds before this
fiddling and about 2 seconds after.

So best guess is that test_longexp + pymalloc-all-the-time tickles some
pathology in your platform's malloc/realloc/free (or did).  Do you have a
lot of RAM?  test_longexp is a memory hog regardless.