string concatentation

[Chris Gonnerman]

----- Original Message -----
From: "Tony Rentschler" <tonyr at walterr.bellatlantic.net>
Subject: string concatentation


> I'm writing a program that reads individual lines of text from stdin,
which
> is filled by piping the output from another program into mine. I need
> to collect the individual lines I've read into one large string. My first
> thought was to do something like this: big_string = big_string +
new_string.
>
> In practice, will this be inefficient? That is, will Python malloc a new
> big_string each time I add a line? Is there another way to handle this
sort
> of situation, say with a pre-allocated buffer of some kind?

Desperately inefficient.  Yes, you get a new malloc() and a free() also
(which
may be deferred but will eventually happen).  The most common idiom for
handling
many lines of input is like so:

# code is 1.5.2 compatible, update to 2.0/2.1 if you really care

import string, sys

line_list = []
input_line = sys.stdin.readline()

while input_line:
    line_list.append(input_line)
    input_line = sys.stdin.readline()

entire_file = string.join(line_list, "")

This will get the final result you are wanting... but if reading the file
into
a string is ALL you want to do (that is, you aren't doing something ELSE in
that
loop), do this:

entire_file = sys.stdin.read()

One malloc(), basically.