<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi friends,<br>
<br>
<u><big>efficient string concatenation</big></u> has been a topic in
2004.<br>
Armin Rigo proposed a patch with the name of the subject,<br>
more precisely:<br>
<br>
<i>[Patches] [ python-Patches-980695 ] efficient string
concatenation</i><i><br>
</i><i>on sourceforge.net, on 2004-06-28.</i><i><br>
</i><br>
This patch was finally added to Python 2.4 on 2004-11-30.<br>
<br>
Some people might remember the larger discussion if such a patch
should be<br>
accepted at all, because it changes the programming style for many
of us<br>
from "don't do that, stupid" to "well, you may do it in CPython",
which has quite<br>
some impact on other implementations (is it fast on Jython, now?).<br>
<br>
It changed for instance my programming and teaching style a lot, of
course!<br>
<br>
But I think nobody but people heavily involved in PyPy expected
this:<br>
<br>
Now, more than eight years after that patch appeared and made it
into 2.4,<br>
PyPy (!) still does _not_ have it!<br>
<br>
Obviously I was mislead by other optimizations, and the fact that<br>
this patch was from a/the major author of PyPy who invented the
initial<br>
patch for CPython. That this would be in PyPy as well sooner or
later was<br>
without question for me. Wrong... ;-)<br>
<br>
Yes, I agree that for PyPy it is much harder to implement without
the<br>
refcounting trick, and probably even more difficult in case of the
JIT.<br>
<br>
But nevertheless, I tried to find any reference to this missing
crucial optimization,<br>
with no success after an hour (*).<br>
<br>
And I guess many other people are stepping in the same trap.<br>
<br>
So I can imagine that PyPy looses some of its speed in many
programs, because<br>
Armin's great hack did not make it into PyPy, and this is not loudly
declared<br>
somewhere. I believe the efficiency of string concatenation is
something<br>
that people assume by default and add it to the vague CPython
compatibility<br>
claim, if not explicitly told otherwise.<br>
<br>
----<br>
<br>
Some silly proof, using python 2.7.3 vs PyPy 1.9:<br>
<br>
<blockquote type="cite">$ cat strconc.py <br>
#!env python<br>
<br>
from timeit import default_timer as timer<br>
<br>
tim = timer()<br>
<br>
s = ''<br>
for i in xrange(100000):<br>
s += 'X'<br>
<br>
tim = timer() - tim<br>
<br>
print 'time for {} concats = {:0.3f}'.format(len(s), tim)<br>
</blockquote>
<br>
<blockquote type="cite">$ python strconc.py <br>
time for 100000 concats = 0.028<br>
$ pypy strconc.py <br>
time for 100000 concats = 0.804<br>
</blockquote>
<br>
Something is needed - a patch for PyPy or for the documentation I
guess.<br>
<br>
This is not just some unoptimized function in some module, but it is
used<br>
all over the place and became a very common pattern since
introduced.<br>
<br>
<i>How ironic that a foreseen problem occurs _now_, and _there_ :-)</i><i><br>
</i><br>
cheers -- chris<br>
<br>
<br>
(*)<br>
<a class="moz-txt-link-freetext" href="http://pypy.readthedocs.org/en/latest/cpython_differences.html">http://pypy.readthedocs.org/en/latest/cpython_differences.html</a><br>
<a class="moz-txt-link-freetext" href="http://pypy.org/compat.html">http://pypy.org/compat.html</a><br>
<a class="moz-txt-link-freetext" href="http://pypy.org/performance.html">http://pypy.org/performance.html</a><br>
<pre class="moz-signature" cols="72">--
Christian Tismer :^) <a class="moz-txt-link-rfc2396E" href="mailto:tismer@stackless.com"><mailto:tismer@stackless.com></a>
Software Consulting : Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121 : *Starship* <a class="moz-txt-link-freetext" href="http://starship.python.net/">http://starship.python.net/</a>
14482 Potsdam : PGP key -> <a class="moz-txt-link-freetext" href="http://pgp.uni-mainz.de">http://pgp.uni-mainz.de</a>
phone +49 173 24 18 776 fax +49 (30) 700143-0023
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? <a class="moz-txt-link-freetext" href="http://www.stackless.com/">http://www.stackless.com/</a></pre>
</body>
</html>