Jack Jansen wrote:
Don't worry, Vladimir, I hadn't forgotten your malloc stuff:-)
Me? worried about mallocs? :-)
if mallopt is available in the standard C library this may be a way to squeeze out a couple of extra percent of performance that the admin who installs Python needn't be aware of.
As long as you're maintaining a Mac-specific port of Python, you can do this without pbs on the Mac port.
And I don't think your allocator can be dropped in to the standard distribution, because it has the potential problem of fragmenting the heap due to multiple malloc packages in one address space (at least, that was the problem when I last looked at it, which is admittedly more than a year ago).
Things have changed since then. Mainly on the Python side. Have a look again.
And about mallopt not being portable: right, but I would assume that something like #ifdef M_MXFAST mallopt(M_MXFAST, xxxx); #endif shouldn't do any harm if we set xxxx to be a size that will cause 80% or so of the python objects to fall into the M_MXFAST category
Which is exactly what pymalloc does, except that this applies for > 95% of all allocations.
(sizeof(PyObject)+sizeof(void *), maybe?). This doesn't sound platform-dependent...
Indeed, I also use this trick to tune automatically the object allocator for 64-bit platforms. I haven't tested it on such machines as I don't have access to them, though. But it should work.
Similarly, M_FREEHD sounds like it could speed up Python allocation, but this would need to be measured. Python allocation patterns shouldn't be influenced too much by platform, so again if this is good on one platform it is probably good on all.
I am against any guesses in this domain. Measures and profiling evidence: that's it. Being able to make lazy decisions about Python's mallocs is our main advantage. Anything else is wild hype <0.3 wink>. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252