[Tutor] Alternatives to append() for "growing" a list

[spir]

On 12/01/2013 05:32 AM, Amit Saha wrote:
> Hello,
>
> I was told by someone (as a comment) that a code snippet such as this
> "would make Pythonistas talk my ear off about how evil the append()"
> function is:
>
>>>> mylist = []
>>>> mylist.append(1)
> # a number of times over
>
> I have some ideas that on an append() the list's internal size
> increases (doubled?) since CPython over-allocates memory, and such.
>
> So, the question is: what is the alternative to growing a list when I
> have no idea what items or how many may be there?

Maybe you are confusing with catenating _strings_, rather than lists. Python's 
concat is problematic because it is a binary operation. So, catenating n bits 
makes n-1 operations, each with intermediate results, of growing sizes, all 
thrown away except for the last one, the actual result. If all of the bitN are 
strings:
	bit1 + bit2 + bit3 + bit4 + bit5
actually constructs:
	bit1+bit2
	bit1+bit2+bit3
	bit1+bit2+bit3+bit4
	bit1+bit2+bit3+bit4+bit5
A number of unneeded string object, and a very big useless memory weight.

Example Python code showing good/bad usage below (advanced pythonistas, please 
correct if needed, I don't know the latest nice Python idioms):

========================================================

# === case of a series of explicite text sections ====

# don't do this:
text = ""

intro = "intro..."
text += intro

body = "body..."
text+= '\n' + body

concl = "concl..."
text += '\n' + concl

print(text) ; print()

# also do not do:
text = intro + '\n' + body + '\n' + concl
print(text) ; print()

# do that...:
intro = "intro..."
body  = "body..."
concl = "concl..."
text = '\n'.join([intro, body, concl])	   # but creates a list
print(text) ; print()

# ...or that:
text = "%s\n%s\n%s" % (intro, body, concl) # no such list
print(text) ; print()

# === case of a list of sections of arbitrary size ====

# (simulation code for demo)
def make_section (n): return "%s\n" % n
section_data = [13, 5, 79, 4, 268, 0, 987654321]

# don't do this:
text   = ""
for dat in section_data:
     section = make_section(dat)
     text += section
print(text)

# do that...:
sections = []
for dat in section_data:
     section = make_section(dat)
     sections.append(section)
text = ''.join(sections)
print(text)

# ... or that:
sections = (make_section(dat) for dat in section_data)
text = ''.join(sections)
print(text)

# ... or even that:
text = ''.join(make_section(dat) for dat in section_data) # no need for ((...))
print(text)