Lazy file.readlines()?

[Alex]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


The fileinput module does something like that in the background, I
think...

I would check out fileinput first, but if that doesn't suit you, you
could use what I rolled for myself a little while back.  I think it's
slightly easier to have multiple instances this way:

class lazy_file:
  def __init__ (self, filename, sizehint = 10**7, mode = 'r'):
    self.file = open (filename, mode)
    self.line_count=0
    self.sizehint = sizehint
    self.__refresh_buffer ()
  def __refresh_buffer (self):
    self.buffer = self.file.readlines (self.sizehint)
    self.buffer_length = len (self.buffer)
    if self.buffer:self.buffer_index = 0
    else: self.buffer_index = None
  def readline (self):
    if self.buffer_index==self.buffer_length:
      self.__refresh_buffer ()
    if not self.buffer: self.line = ''
    else:
      self.line = self.buffer[self.buffer_index]
      self.buffer_index = self.buffer_index + 1
    if self.line:self.line_count=self.line_count+1
    return self.line

class File_loop:
  def __init__(self,filename,sizehint=10**7,mode='r'):
    self.file=lazy_file(filename,sizehint,mode)
  def __getitem__(self,index):
    if self.file.readline():return self.file.line
    else:raise IndexError, 'File finished'

You can just use it like

for line in File_loop('filename'):
  munge(line)

Alex.

- -- 
If cars were like computers, they would go 300 m.p.h. and get a hundred
miles to the gallon and cost $50. Except that twice a month someone a
thousand miles away would be able to blow up the car, killing everyone
nearby.

-----BEGIN PGP SIGNATURE-----
Version: PGP for Personal Privacy 5.0
Comment: Processed by Mailcrypt 3.5b6, an Emacs/PGP interface
Charset: noconv

iQA/AwUBN+BT93zN4ZFYpUPAEQJQFACgrZ2asMAo6PPw7He+yVZsXd7qyHQAoMOg
7szYmUaTVubWw0fXZMWxdjeh
=6uiN
-----END PGP SIGNATURE-----