A really bad idea.

[Gonçalo Rodrigues]

On Thu, 14 Nov 2002 15:33:33 -0800, "Russell E. Owen"
<owen at nospam.invalid> wrote:

>In article <c0abefc3.0211140738.635c0e6d at posting.google.com>,
> marvelan at hotmail.com (M) wrote:
>

[text snipped]

>
>As a concept, I feel generators and iterators are well worth learning 
>and using. I make pretty heavy use of them and am glad to be saving 
>myself the code I'd otherwise have to write. On the other hand:
>- There is some area of all this I still find confusing. it's hard to 
>put my finger on, but I'm not always sure whether I should be obtaining 
>an iterator and then using it in a for loop or defining __iter__ (or is 
>it iter?) and using my object directly in a for loop. Also, I'm not 
>always sure about iterators or generators that call iterators or 
>generators.

Let me try and shoot at the fog and see if I can clear it up. First a
definition: An object <whatever> is iterable (or is an iterable) if

iter(<whatever>)

returns *something* (and does not raise an exception). Iterables or nice
because they allow for loops, e.g.

for elem in <iterable>:
    <whatever>

Examples: list, tuple, file, etc.
>>> print iter([])
<iterator object at 0x01126098>
>>> 

Now iter() is a builtin function and when called on an iterable it
returns an *iterator* which may or may not be the same thing as the
iterable itself.

>>> a = []
>>> print id(a), id(iter(a))
17990976 17999536

As you can see the list iterable is different from the iterator itself.
I think (but I am not sure) that files are scheduled in 2.3 to become
their own iterators. Anyway, what the iterator object is supposed to
encapsulate is the traversal through the elements of the iterable. It
does this by having a method, next(), that gives the next element
(whatever that may be) or raises an exception if there is nothing left
to spew.

When Python encounters a

for elem in <iterable>:
    <whatever>

it basically is the same: get the iterator of iterable and sucessively
bind elem to whatever the iterator's next method returns until a
StopIteration exception is raised.

Note also that an iterator is also an iterable. But the iterator of an
iterator is the iterator itself (usually - I suppose there is some more
exotic code out there where this rule is broken).

>>> b = iter(a)
>>> print b
<iterator object at 0x01129C68>
>>> print b is iter(b)
1

If you want to code your own iterable just define __iter__. If it is an
iterator make __iter__ return self (unless you know what you are doing)
and provide a next method.

Now for generators. Generators are a cross-breeding between a function
and an iterable. It is better an example first:

>>> def test(n = 0):
... 	start = n
... 	while True:
... 		yield start
... 		start += 1
... 		
>>> print test
<function test at 0x01144A88>

As you can see test *is* a function. When called for the first time it
gives

>>> a = test()
>>> print a
<generator object at 0x01148DA0>

A generator. Now a generator *is* an iterator. To see that:

>>> a.next
<method-wrapper object at 0x010D0FD0>
>>> print id(a), id(iter(a))
18124192 18124192

But it is a special kind of iterator. It remembers all the state about
the "function". Let us go back to our little example to see how it
works. 

When you first call a.next() (either explicitely or implicitely in a for
loop) it executes the code in test until the first yield. When it gets
there it "returns" whatever expression is in front of the yield but
instead of discarding the function it "suspends" it. This means two
things (at least): The state of all the local variables is kept **and**
the place where you yielded is also kept. On the next next() call you
execute the body of test with your local variables in the state they
were in *and* right from the place where you last yielded: In our case
you start with

start += 1

which bumps start to 1.

And that is basically all there is to it.

>
>-- Russell

All the best,
G. Rodrigues

P.S: Gonna go back to 'Finnegans Wake' to see if I can improve my
english style. Ach, it sucks bad!