[Python-ideas] Performance of the "".join() idiom

[Antoine Pitrou]

For the record, the "".join() idiom also has its downsides. If you
build a list of many tiny strings, memory consumption can grow beyond
the reasonable (in one case, building a 600MB JSON string outgrew the
RAM of an 8GB machine). One solution is to regularly accumulate the
primary list into a secondary accumulation list as done in
http://hg.python.org/cpython/rev/47176e8d7060

Regards

Antoine.




On Thu, 25 Aug 2011 12:28:14 +0300
k_bx <k.bx at ya.ru> wrote:

> Hi!
> 
> There's a certain problem right now in python that when people need to build string from pieces they really often do something like this::
> 
>     def main_pure():
>         b = u"initial value"
>         for i in xrange(30000):
>             b += u"more data"
>         return b
>