File reading using delimiters

[Tim Evans]

kylotan at hotmail.com (Kylotan) writes:

> All the examples of reading files in Python seem to concern reading a
> line at a time. But this is not much good to me as I want to be able
> to read up to arbitrary delimiters without worrying about how many
> lines I'm spanning. With my rudimentary Python knowledge I'm having to
> read in multiple lines, concatenate them, search for the delimiter,
> split the result if necessary, and carry forward whatever was after
> the delimiter to the next operation. Is there a better way of reading
> until a certain character is encountered, and no more?

The following code avoids both reading the whole file in one go, or
reading one character at a time:



from __future__ import generators

def readdelim(f, delim, blocksize=1024):
    end = ''
    while 1:
        block = f.read(blocksize)
        if not block:
            break
        parts = block.split(delim)
        if len(parts) == 1:
            end += parts[0]
        else:
            yield end + parts[0]
            for p in parts[1:-1]:
                yield p
            end = parts[-1]
    yield end



if __name__ == '__main__':
    import cStringIO as StringIO

    f = StringIO.StringIO('foo:bar:blarg:wibble:spam:eggs:parrot')
    for line in readdelim(f, ':', 10):
        print `line`



-- 
Tim Evans