[Python-Dev] PATCH submitted: Speed up + for string concatenation, now as fast as "".join(x) idiom

[Larry Hastings]

Gregory P. Smith wrote:
> have you run any generic benchmarks such as pystone to get a better
> idea of what the net effect on "typical" python code is?
I hadn't, but I'm happy to.  On my machine (a fire-breathing Athlon 64 
x2 4400+), best of three runs:

Python 2.5 release:
    Pystone(1.1) time for 50000 passes = 1.01757
    This machine benchmarks at 49136.8 pystones/second

Python 2.5 concat:
    Pystone(1.1) time for 50000 passes = 0.963191
    This machine benchmarks at 51910.8 pystones/second

I'm surprised by this; I had expected it to be slightly *slower*, not 
the other way 'round.  I'm not sure why this is.  A cursory glance at 
pystone.py doesn't reveal any string concatenation using +, so I doubt 
it's benefiting from my speedup.  And I didn't change the optimization 
flags when I compiled Python, so that should be the same.


Josiah Carlson wrote:
> Regardless of "nicer to read", I would just point out that Guido has
> stated that Python will not have strings implemented as trees.
>   
I suspect it was more a caution that Python wouldn't *permanently* store 
strings as "ropes".  In my patch, the rope only exists until someone 
asks for the string's value, at which point the tree is rendered and 
dereferenced.  From that point on the object is exactly like a normal 
PyStringObject to the external viewer.

But you and I are, as I believe the saying goes, "channeling Guido 
(badly)".  Perhaps some adult supervision will intervene soon and make 
its opinions known.


For what it's worth, I've realized two things I want to change about my 
patch:

  * I left in a couple of /* lch */ comments I used during development 
as markers to find my own code.  Whoops; I'll strip those out.

  * I realized that, because of struct packing, all PyStringObjects are 
currently wasting an average of two bytes apiece.  (As in, that's 
something Python 2.5 does, not something added by my code.)  I'll change 
my patch so strings are allocated more precisely.  If my string 
concatenation patch is declined, I'll be sure to submit this patch 
separately.

I'll try to submit an updated patch today.

Cheers,


/larry/