memory usage multi value hash

[Terry Reedy]

On 4/14/2011 12:55 PM, Peter Otten wrote:

> I don't expect that it matters much, but you don't need to sort your data if
> you use a dictionary anyway:

Which means that one can build the dict line by line, as each is read, 
instead of reading the entire file into memory. So it does matter for 
intermediate memory use.

> import csv
> import sys
>
> infile, outfile = sys.argv[1:]
>
> d = {}
> with open(infile, "rb") as instream:
>      for key, value in csv.reader(instream, delimiter=';'):
>          d.setdefault(key, [key]).append(value)
>
> with open(outfile, "wb") as outstream:
>      csv.writer(outstream).writerows(d.itervalues())


-- 
Terry Jan Reedy