Fast way to grab line from file?

Alex alex at somewhere.round.here
Mon Feb 21 21:27:41 EST 2000


Hi, Mateo.

You try something like this:

class lazy_file:
    
    def __init__ (self,filename, sizehint=10**7):
        
        self.file = open (filename, mode)
        self.sizehint = sizehint
        self.buffer = None
        
    def readline (self):
        
        if not self.buffer:
            self.buffer = self.file.readlines (self.sizehint)
        if not self.buffer:
            self.line = ''
        else:
            self.line = self.buffer.pop (0)
        return self.line

f = lazy_file('file_i_want_line_100_from')

for i in 100*[None]:
    line = f.readline ()

I don't know why, but it seems that slurping up a chunk of the file
using readlines and a sizehint is generally faster than just doing a
whole lot of readline calls.  It may help in your case.

If that still weren't fast enough, I would probably try something like 

first_hundred = os.popen('head -100 file_i_want_line_100_from').readlines()
if len(first_hundred) == 100:
    line = first_hundred[-1]
else:
    line = ''

next.  Don't really know whether it would help, though.

Alex.



More information about the Python-list mailing list