Python IS slow ! [was] Re: Python too slow for real world

[Christian Tismer]

Tim Peters wrote:
> 
> [various, about malloc/free performance]
> 
> [Christian Tismer]
> > ...
> > A malloc just for Python - would it need locking at all?
> > Since the interpreter is locking already, I guess malloc
> > would be always called in a safe way.
> 
> Right.  Note that Vladimir has already put a great deal of work into this
> area; see
> 
>     http://starship.python.net/crew/vlad/pymalloc/
> 
> IIRC, paradoxically this yielded bigger speedups on non-Windows systems.
> 
> windows-vm-flushes-most-recently-used<0.9-wink>-ly y'rs  - tim

Thanks a lot. Now I have so many ways to go that I cannot
decide. Vlad uses explicit bins, dedicated storage. While
this can be very fast, it is a generalization of Guido's
pre-allocated arrays for some kinds of objects. Nobody
can tell me if this is the way to go, or if it will
under circumstances yield much fragmentation. Doug Lea's
code is more general, since his bins are linked lists of
free space which can coalesce again. Takes more time and
less fragmentation. And J.C. Wippler gave me a very fast
allocator for small objects which uses a power of two
size scheme, with memory blocks arranged in a way that
their size is encoded in the address (portable!).

I guess finding out "the right way"(TM) is NP-complete.

If I knew in advance how long certain objects will live,
then I would be able to put them into the proper place.
Does it make sense to do statistics on object lifetime?
But the only known thing is allocation size.

Also, Python knows a lot more than malloc: A list(dict)
is always two things: A header which cannot move, and
a body which can move and grow and shrink. Could make
sense to use different strategies for heads and bodies.
Bad again with strings and tuples which have varying
size and are not moveable. I would be much happier if
we would pay the one level of indirection, but we
had all the varying sizes moveable, since this would
allow for some compaction as well. 

But I know it is too late :-)

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at appliedbiometrics.com>
Applied Biometrics GmbH      :     Have a break! Take a ride on Python's
Kaiserin-Augusta-Allee 101   :    *Starship* http://starship.python.net
10553 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     we're tired of banana software - shipped green, ripens at home