FileInput too slow
wiso
gtu2003 at alice.it
Mon Jan 4 17:35:02 EST 2010
I'm trying the fileinput module, and I like it, but I don't understand why
it's so slow... look:
from time import time
from fileinput import FileInput
file = ['r1_200907.log', 'r1_200908.log', 'r1_200909.log', 'r1_200910.log',
'r1_200911.log']
def f1():
n = 0
for f in file:
print "new file: %s" % f
ff = open(f)
for line in ff:
n += 1
ff.close()
return n
def f2():
f = FileInput(file)
for line in f:
if f.isfirstline(): print "new file: %s" % f.filename()
return f.lineno()
def f3(): # f2 simpler
f = FileInput(file)
for line in f:
pass
return f.lineno()
t = time(); f1(); print time()-t # 1.0
t = time(); f2(); print time()-t # 7.0 !!!
t = time(); f3(); print time()-t # 5.5
I'm using text files, there are 2563150 lines in total.
More information about the Python-list
mailing list