[Tutor] Iterators and Generators...

[Neil Schemenauer]

Marc Barry wrote:
> I have a question, why didn't Python just cause a method with a 'yield' 
> statement to return its value and preserve its state.  Then, upon 
> subsequent calls to the method it could begin execution from the point it 
> left off.  Basically, I am asking why they didn't choose the above option 
> and chose to return a generator (i.e. build the object for you) instead?  
> With the above option, a person could easily implement the iterator 
> protocol and there is no need for a generator.

An iterator is essentially an object that remembers the state of the
iteration.  Where should this state be stored?  For a list, the thing to
remember is the current index and the length of the list.  Each time you
want to iterate over a list you get a new iterator.  Witness:

    >>> seq = ["a", "b", "c"]
    >>> it1 = iter(seq)
    >>> it2 = iter(seq)
    >>> it1
    <listiterator object at 0x4025666c>
    >>> it2
    <listiterator object at 0x4025676c>
    >>> it1.next()
    'a'
    >>> it2.next()
    'a'
    >>> it1.next()
    'b'
    >>> it2.next()
    'b'
    >>> it1.next()
    'c'
    >>> it1.next()
    Traceback (most recent call last):
      File "<stdin>", line 1, in ?
    StopIteration

The list cannot be it's own iterator (hopefully that's obvious).  You
would not be able to iterate over the same list concurrently.  For
example:

    seq = ["a", "b", "c"]
    for x in seq:
        for y in seq:
            print x, y

While the inner 'for' loop is executing there are two iterators alive.
I hope this is a good clue as to why generator-functions return a
generator-iterator when called.  For all the details on iterators you
can read the PEP:

    http://www.python.org/peps/pep-0234.html

You can skip the C API part.  There is also a PEP for generators, if you
are interested.

  Neil