[Tutor] Iterators and generators for the masses

[dman]

On Tue, Feb 26, 2002 at 02:40:45AM +0900, kevin parks wrote:

| Iterators (PEP 234) and generators (PEP 255) were added.

| what do they do? what can they do, what tasks do they make easier,
| what is the simplest possible example, what is a more complex
| example...etc. & co.

Iteration is the process of performing some operation on each member
of a collection of objects.  For example, to iterate over a list and
simply print out each element, you would use

    for item in the_list :
        print str( item )

List objects follow the iterator protocol.  This simply means they
define a method that when called returns an object that follows a
certain interface.  The interface is that the next() method returns
the "next" object in the collection.  The iterator itself determines
what the "next" object is.  (suppose you were iterating over a tree,
do you want it depth-first or breadth-first and pre-order, in-order,
or post-order?)

Iterators can be plugged directly into the 'for' syntax, or you can
use them manually by calling next() when you want the next item.

Generators are simply some convenient syntax-sugar for creating an
iterator.  A generator is a function that, when called, returns an
iterator object.  The iterator object (as described above) provides
each piece of data that the generator generates, in order.  For
example, the most common use of xrange can be written as a generator :

def my_xrange( limit ) :
    cursor = 0  # the first element
    while cursor < limit :
        yield cursor
        cursor += 1

This generator simply generates a sequence of integers starting with 0
and ending at floor(limit-1).  I can then use this generator just as I
would use any other iterable object.

>>> for i in my_xrange( 10 ) :
...     print i ,
...
0 1 2 3 4 5 6 7 8 9
>>>

The neat part is that I don't have to create an actual sequence.  If
the limit was 1,000,000 the memory usage is the same as in this
example.  I don't even have to create a class with a magic __getitem__
method to compute the next item in the sequence.  The generator is
written quite naturally (think of 'yield' much like 'return', except
that execution will resume at the following line).

This is iterators and generators in a nutshell.  If you have more
specific questions, fire away.

-D

-- 

If anyone would come after me, he must deny himself and take up his
cross and follow me.  For whoever wants to save his life will lose it,
but whoever loses his life for me and for the gospel will save it.  What
good is it for a man to gain the whole world, yet forfeit his soul?  Or
what can a man give in exchange for his soul?
        Mark 8:34-37