[pypy-issue] [issue1268] file.writelines() very slow on Windows

[Simon Lundberg]

New submission from Simon Lundberg <simon.lundberg at yahoo.com>:

Using file.writelines() in 32-bit PyPy on Windows seems very slow (40x-60x 
slower than in CPython). Using file.write() with large chunks of data is much 
faster (though still not as fast as in CPython).

Attached is a small function that takes a file path and number of lines in the 
file as arguments. It creates a set of dummy data, and writes it out using 
file.write() and file.writelines(), and prints out the timings. I'm getting the 
following results:

CPython 64-bit:
f.write(): 0.098
f.writelines(): 0.161

CPython 32-bit:
f.write(): 0.127
f.writelines(): 0.205

PyPy 32-bit:
f.write(): 0.297
f.writelines(): 9.47

----------
files: writeTest.py
messages: 4782
nosy: SimonLundberg, pypy-issue
priority: performance bug
status: unread
title: file.writelines() very slow on Windows

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1268>
________________________________________
-------------- next part --------------
import time

def writeTest(path, numLines=1000000):
    t = [time.time()]
    data = '\n'.join(['this is a line of data' for i in range(numLines)])
    t.append(time.time())
    print 'Creating data:  ', t[-1] - t[-2]
    with file(path, 'w') as f: f.write(data)
    t.append(time.time())
    print 'f.write():      ', t[-1] - t[-2]

    data = ['this is a line of data\n' for i in range(numLines)]
    t.append(time.time())
    print 'Creating data:  ', t[-1] - t[-2]
    with file(path, 'w') as f: f.writelines(data)
    t.append(time.time())
    print 'f.writelines(): ', t[-1] - t[-2]