[Python-Dev] Re: Sets: elt in dict, lst.include

[Guido van Rossum]

> [Tim Peters on adding yet more syntatic sugar]
> > Available time is finite, and this isn't at the top of the list
> > of things I'd like to see (resuming the discussion of
> > generators + coroutines + iteration protocol comes to mind
> > first).
> 
> What's the chances of getting generators into 2.2?  The
> implementation should not be hard.  Didn't Steven Majewski have
> something years ago?  Why do we always get sidetracked on trying
> to figure out how to do coroutines and continuations?

I think there's a very good chance of getting them into 2.2.  But it
*is* true that coroutines are a very attractice piece of land "just
nextdoor".  On the other hand, continiations are a mirage, so don't
try to go there. :-)

> Generators would add real power to the language and are simple
> enough that most users could benefit from them.  Also, it should be
> possible to design an interface that does not preclude the
> addition of coroutines or continuations later.
> 
> I'm not volunteering to champion the cause just yet.  I just want
> to know if there is some issue I'm missing.

There are different ways to do interators.

Here is a very "tame" proposal (and definitely in the realm of 2.2),
that doesn't require any coroutine-like tricks.  Let's propose that

    for var in expr:
	...do something with var...

will henceforth be translated into

    __iter = iterator(expr)
    while __iter.more():
        var = __iter.next()
        ...do something with var...

-- or some variation that combines more() and next() (I don't care).

Then a new built-in function iterator() is needed that creates an
iterator object.  It should try two things:

(1) If the object implements __iterator__() (or a C API equivalent),
    call that and be done; this way arbitrary iterators can be
    created.

(2) If the object smells like a sequence (how to test???), use an
    iterator sort of like this:

    class Iterator:

        def __init__(self, sequence):
            self.sequence = sequence
            self.index = 0

        def more(self):
	    # Store the item so that each index is tried exactly once
            try:
                self.item = self.sequence[self.index]
            except IndexError:
                return 0
            else:
                self.index = self.index + 1
                return 1

        def next(self):
            return self.item

    (I don't necessarily mean that all those instance variables should
    be publicly available.)

The built-in sequence types can use a very fast built-in iterator type
that uses a C int for the index and doesn't store the item in the
iterator.  (This should be as fast as Marc-Andre's for loop
optimization using a C counter.)

Dictionaries can define an appropriate iterator that uses
PyDict_Next().

If the argument to iterator() is itself an iterator (how to test???),
it returns the argument unchanged, so that one can also write

    for var in iterator(obj):
	...do something with var...

Files of course should have iterators that return the next input line.

We could build filtering and mapping iterators that take an iterator
argument and do certain manipulations with the elements; this would
effectively introduce the notion lazy evaluation on sequences.

Etc., etc.

This does not come close to Icon generators -- but it doesn't require
any coroutine-like capabilities, unlike those.

--Guido van Rossum (home page: http://www.python.org/~guido/)