Python too slow for real world

[Dan Schmidt]

Arne Mueller <a.mueller at icrf.icnet.uk> writes:

| I can't read in the whole file as a single block, it's too big, if
| readline/write is slow the program will never get realy fast :-(

You can use the 'sizehint' parameter to readlines() to get some of the
efficiency of readlines() without reading in the whole file.  The
following code isn't optimized, but it shows the idea:

class BufferedFileReader:
    def __init__ (self, file):
        self.file = file
        self.lines = []
        self.numlines = 0
        self.index = 0

    def readline (self):
        if (self.index >= self.numlines):
            self.lines = self.file.readlines(65536)
            self.numlines = len(self.lines)
            self.index = 0
            if (self.numlines == 0):
                return ""
        str = self.lines[self.index]
        self.index = self.index + 1
        return str

-- 
                 Dan Schmidt -> dfan at harmonixmusic.com, dfan at alum.mit.edu
Honest Bob & the                http://www2.thecia.net/users/dfan/
Factory-to-Dealer Incentives -> http://www2.thecia.net/users/dfan/hbob/
          Gamelan Galak Tika -> http://web.mit.edu/galak-tika/www/