Reading variable length records...

[Oleg Broytmann]

On Wed, Sep 12, 2001 at 04:21:47PM -0600, Bjorn Pettersen wrote:
> I'm trying to read records from a 2 GB datafile, but my brain has
> stopped working, so I was wondering if someone has allready solved this
> problem. The records are variable length and are separated by a five
> character delimiter. I was trying to use file.read(n) with a blocksize
> of ~1Mb, but got a serious brainfart when trying to think of how to
> handle the case where only part of the delimiter was read in the current
> block.

   The pattern to do it is called "sliding window", and I recommend to use
memory mapped files (if it possible) - theys are simpler to manipulate at
this level (and probably faster, as OS will optimize access).

Oleg.
---- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.