Restart generator when it is exhausted.

[J. Cliff Dyer]

On Tue, 2009-04-28 at 10:41 +0000, Duncan Booth wrote: 
> Lacrima <Lacrima.Maxim at gmail.com> wrote:
> 
> > If it is not possible what are common techniques to use iterator or
> > generator objects that allow restarting when it is needed?
> 
> The usual thing if you want to use the generator's output more than once  
> would be to convert the generator to a list, then you can iterate over it 
> as often as you want.
> 
> >>> a = ['a', 'b', 'c']
> >>> g = (i for i in a)
> >>> restartable = list(g)
> 
> If you want the output of the generator to potentially change each time you 
> iterate then you need to create a new generator.
> 

More verbosely, but without putting your generator in , you can use the
iterator protocol to create a reusable iterable:

An iterable is a class with an __iter__ method that returns an iterator.

So for example:

class Iterator(object):
    def __init__(self, filename):
        self.f = open(filename)
    
    def __iter__(self):
        return self

    def next(self):
        line = self.f.readline()
        if not line:
            raise StopIteration
        return line.strip()[:8]

is an iterator (which is also an iterable), which will grab each line of
a file, returning the first eight non-whitespace characters until the
file is used up.  Then the iterator is exhausted, and will continue to
raise StopIteration each time it is called.

class Iterable(object):
    def __init__(self, filename):
        self.filename = filename

    def __iter__(self):
        return Iterator(self.filename)

This is a reusable iterable which returns a new instance of the Iterator
class above each time it is exhausted.

So given a file hello.txt:

  Hello world
    Hola mundo
  Guten tag, weld.

The classes can be used as followed:
>>> iterator = Iterator('hello.txt')
>>> for i in xrange(3):
>>>     print "*** %d ***" % i
>>>     for j in iterator:
>>>         print j
*** 0 ***
Hello wo
Hola mun
Guten ta
*** 1 ***
*** 2 ***
>>> iterable = Iterable('hello.txt')
>>> for i in xrange(3):
>>>     print "*** %d ***" % i
>>>     for j in iterable:
>>>         print j
*** 0 ***
Hello wo
Hola mun
Guten ta
*** 1 ***
Hello wo
Hola mun
Guten ta
*** 2 ***
Hello wo
Hola mun
Guten ta

When Iterator hits a StopIteration, it passes out of the inner loop, and
when it comes back in, the inner loop calls iterator.__iter__(), and
gets the same exhausted iterator (which immediately breaks the inner
loop by raising StopIteration).  In Iterable, when the loop calls
iterable.__iter__(), it gets a fresh iterator, so it can loop over the
file again.

The important thing is that when you call x.__iter__() (which you do
when entering a loop), you get a fresh iterator that won't just call
StopIteration right away.

Cheers,
Cliff