[Python-3000] Reversing through text files with the new IO library

Mark Russell mark.russell at zen.co.uk
Sat Mar 10 15:22:52 CET 2007


As an experiment with the new IO library (http://docs.google.com/Doc? 
id=dfksfvqd_1cn5g5m), I added support for walking through the lines  
of a file backwards using the reversed() iterator:

     for line in reversed(open(path)):
          ...

It works by scanning backwards through the file a block at a time  
(using seek()) but will handle arbitrary length lines.

The patch is at http://www.python.org/sf/1677872.  The code is  
currently directly in Lib/io.py, but in fact the only internal access  
it needs is adding the __reversed__ hook to TextIOWrapper.

It's useful for scanning backwards through large log files, but it's  
also IMHO a nice example of the benefits of the new library.  The  
version of this that I used under python 2.5 had to use os.seek() etc  
on file descriptors, whereas now it just wraps a new buffering class  
(io.TextLineReverser) around the raw IO object.  Among other things  
it makes unit tests simpler - instead of messing around with  
temporary files the tests can do things like:

     b = io.BytesIO(b'one\ntwo\nthree\n')
     assert list(io.TextLineReverser(b)) == [ 'three\n', 'two\n', 'one 
\n' ]

Mark Russell



More information about the Python-3000 mailing list