[Tutor] large strings and garbage collection

[Rich Lovely]

2009/7/17 Dinesh B Vadhia <dineshbvadhia at hotmail.com>:
> This was discussed in a previous post but I didn't see a solution.  Say, you
> have
>
> for i in veryLongListOfStringValues:
>     s += i
>
> As per previous post
> (http://thread.gmane.org/gmane.comp.python.tutor/54029/focus=54139),
> (quoting verbatim) "... the following happens inside the python interpreter:
>
> 1. get a reference to the current value of s.
> 2. get a reference to the string value i.
> 3. compute the new value += i, store it in memory, and make a reference to
> it.
> 4. drop the old reference of s (thus free-ing "abc")
> 5. give s a reference to the newly computed value.
>
> After step 3 and before step 4, the old value of s is still referenced by s,
> and the new value is referenced internally (so step 5 can be performed). In
> other words, both the old and the new value are in memory at the same time
> after step 3 and before step 4, and both are referenced (that is, they
> cannot be garbage collected). ... "
>
> As s gets very large, how do you deal with this situation to avoid a memory
> error or what I think will be a general slowing down of the system if the
> for-loop is repeated a large number of times.
>
> Dinesh
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> http://mail.python.org/mailman/listinfo/tutor
>
>

If all you are doing is concatenating a list of strings, use the
str.join() method, which is designed for the job:

>>> listOfStrings
['And', 'now', 'for', 'something', 'completely', 'different.']
>>> print " ".join(listOfStrings)
And now for something completely different.
>>> print "_".join(listOfStrings)
And_now_for_something_completely_different.

If you need to perform other operations first, you can pass a
generator expression as the argument, for example:

>>> " ".join((s.upper() if n%2 else s.lower()) for n, s in enumerate(listOfStrings))
'and NOW for SOMETHING completely DIFFERENT.'


Hope that helps you.
-- 
Rich "Roadie Rich" Lovely
There are 10 types of people in the world: those who know binary,
those who do not, and those who are off by one.