[pypy-issue] [issue1429] CSV module is slow

[mattip]

mattip <matti.picus at gmail.com> added the comment:

PyPY on Windows is still slow, but the problem seems to be reading from the 
file. By using the csvtest_preload.py to read the file into a list of strings, 
the times improve greatly. This points to IO problems with Windows more than 
parsing problems with CSV

csvtest.py -
reading took 1.986 seconds
writing took 0.411 seconds
complete took 2.397 seconds

csvtest_preload.py
reading took 0.292 seconds
writing took 0.243 seconds
complete took 0.535 seconds

----------
nosy: +mattip
status: resolved -> chatting

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1429>
________________________________________
-------------- next part --------------

import csv
import time

with open('airports.csv') as file_in:
    lines = file_in.readlines()
for i in range(10):
    t0 = time.time()
    rows = list(csv.reader(lines))
    t1 = time.time()
    with open('airports.csv.out', 'wb') as file_out:
        writer = csv.writer(file_out)
        writer.writerows(rows)
    t2 = time.time()

    print 'reading took %.3f seconds' % (t1 - t0)
    print 'writing took %.3f seconds' % (t2 - t1)
    print 'complete took %.3f seconds' % (t2 - t0)