[Tutor] Fastest way to iterate through a file
sigzero at gmail.com
Tue Jun 26 16:16:44 CEST 2007
Kent Johnson wrote:
> Robert Hicks wrote:
>> This is the loop code:
>> for line in f2:
>> for id in idList:
>> if id in line:
>> print "%s: %s" % (id, f2.next())
>> found = "%s: %s" % (id, f2.next())
>> I have an list, idList, that contains a list of id numbers. That code
>> will loop the the f2 file and for lines that have an id on it it will
>> print the "next" line (so I can see what it is doing) and write it to a
>> file. I will turn off that screen print after I get it going the way I
>> want it to.
> I don't see any particular reason this should be slow unless idList is
> large. Perhaps the output is being buffered somewhere and not appearing
> until the process is done? How are you running the program?
> BTW the line that you print and the line that you save to the file are
> not the same; every time you call f2.next() it will fetch another line
> from the file. If you want them to be the same you could say
> found = "%s: %s" % (id, f2.next())
> print found
Thanks for that!
idList only has about 129 id numbers in it.
I am running it straight from a Linux console. I thought about buffering
but I am not sure how Python handles that.
Do you know if Python has a "slower" startup time than Perl? That could
be part of it though I suspect the buffering thing more.
More information about the Tutor