mallopt (Re: [Python-Dev] Minor compilation problem on HP-UX

[Vladimir Marangozov]

Jack Jansen wrote:
> 
> Don't worry, Vladimir, I hadn't forgotten your malloc stuff:-)

Me? worried about mallocs? :-)

> if mallopt is available in the standard C library this may be a way
> to squeeze out a couple of extra percent of performance that the admin
> who installs Python needn't be aware of.

As long as you're maintaining a Mac-specific port of Python, you can
do this without pbs on the Mac port.

> And I don't think your allocator can be dropped in 
> to the standard distribution, because it has the potential problem of 
> fragmenting the heap due to multiple malloc packages in one address
> space (at least, that was the problem when I last looked at it, which
> is admittedly more than a year ago).

Things have changed since then. Mainly on the Python side.
Have a look again.

> 
> And about mallopt not being portable: right, but I would assume that
> something like
> #ifdef M_MXFAST
> 	mallopt(M_MXFAST, xxxx);
> #endif
> shouldn't do any harm if we set xxxx to be a size that will cause 80%
> or so of the python objects to fall into the M_MXFAST category 

Which is exactly what pymalloc does, except that this applies for > 95% of
all allocations.

> (sizeof(PyObject)+sizeof(void *), maybe?). This doesn't sound 
> platform-dependent...

Indeed, I also use this trick to tune automatically the object allocator
for 64-bit platforms. I haven't tested it on such machines as I don't have
access to them, though. But it should work.

> Similarly, M_FREEHD sounds like it could speed up Python allocation,
> but this would need to be measured. Python allocation patterns shouldn't
> be influenced too much by platform, so again if this is good on one
> platform it is probably good on all.

I am against any guesses in this domain. Measures and profiling evidence:
that's it.  Being able to make lazy decisions about Python's mallocs is
our main advantage. Anything else is wild hype <0.3 wink>.

-- 
       Vladimir MARANGOZOV          | Vladimir.Marangozov@inrialpes.fr
http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252