Reading a large csv file

skip at skip at
Wed Jun 24 12:21:57 EDT 2009

    Mag> s=0

    Mag> #Takes the longest here
    Mag> for y in fs:
    Mag>      continue
    Mag>   a=y.split(',')
    Mag>   s=s+1
    Mag>   dset.resize(s,axis=0)
    Mag> fs.close()

    Mag> f.close()

    Mag> This works but just takes a VERY long time.

    Mag> Any way to optimize this?

I sort of suspect you're missing something there.  Is there nothing between
the for loop and the overly indented continue statement?

At any rate, try using the csv module to read in your records:

    import csv

    reader = csv.reader(fs)

    for s, row in enumerate(reader):
        dset.resize(s, axis=0)

Skip Montanaro - skip at -
    when i wake up with a heart rate below 40, i head right for the espresso
    machine. -- chaos @

More information about the Python-list mailing list