[Python-Dev] PEP 340 -- loose ends

[Delaney, Timothy C (Timothy)]

Guido van Rossum wrote:

> [Delaney, Timothy]
>> PEP 340 does not address "normal" iterators very well, but a
>> properly-constructed iterator will behave correctly.
> 
> This is by design.

Yep - I agree.

>> The PEP though is very generator-focussed.
> 
> Disagree. The PEP describes most everything (e.g. the block statement
> semantics) in terms of iterators, and then describes how the new APIs
> behave for generators.

Again, agree. What I meant is that there are no examples of how to
actually implement the correct semantics for a normal iterator. Doing it
right is non-trivial, especially with the __next__ and __exit__
interaction (see below).

>> The issues I see for "normal"
>> iterators (and that need to be addressed/stated in the PEP) are:
>> 
>>     1. No automatic handling of parameters passed to __next__ and
>>        __exit__. In a generator, these will raise at the yield
>>        -statement or -expression. A "normal" iterator will have
>>        to take care of this manually. 
> 
> Not sure what you mean by this. If __next__() is defined, it is passed
> the parameter; if only next() is defined, a parameter (except None) is
> an error. That seems exactly right. Also, if __exit__() isn't defined,
> the exception is raised, which is a very sensible default behavior
> (and also what will happen to a generator that doesn't catch the
> exception).

What I meant is how the iterator is meant to handle the parameters
passed to each method. PEP 340 deals with this by stating that
exceptions will be raised at the next yield-statement or -expression. I
think we need an example though of how this would translate to a
"normal" iterator. Something along the lines of::

    class iterator (object):

        def next (self):
            return self.__next__()

        def __next__(self, arg=None):
            value = None

            if isinstance(arg, ContinueIteration):
                value = arg.value
            elif arg is not None:
                raise arg

            if value is None:
                raise StopIteration

            return value

        def __exit__(self, type=None, value=None, traceback=None):
            if (type is None) and (value is None) and (traceback is
None):
                type, value, traceback = sys.exc_info()

            if type is not None:
                try:
                    raise type, value, traceback
                except type, exc:
                    return self.__next__(exc)
 
           return self.__next__()

>> As another option, it might be worthwhile creating a base iterator
type
>> with "correct" semantics.

> Well, what would the "correct" semantics be? What would passing a
> parameter to a list iterator's __next__() method mean?

Sorry - I meant for user-defined iterators. And the correct semantics
would be something like the example above I think. Except that I think
most of it would need to be in a separate method (e.g. _next) for base
classes to call - then things would change to be something like::

    class iterator (object):
        ...

        def _next (self, arg):
            if isinstance(arg, ContinueIteration):
                return arg.value
            elif arg is not None:
                raise arg

        def __next__(self, arg=None):
            value = self._next(arg)

            if value is None:
                raise StopIteration

            return value            

        ...

Finally, I think there is another loose end that hasn't been addressed::

    When __next__() is called with an argument that is not None, the
    yield-expression that it resumes will return the value attribute
    of the argument.  If it resumes a yield-statement, the value is
    ignored (or should this be considered an error?).  When the
    *initial* call to __next__() receives an argument that is not
    None, the generator's execution is started normally; the
    argument's value attribute is ignored (or should this be
    considered an error?).  When __next__() is called without an
    argument or with None as argument, and a yield-expression is
    resumed, the yield-expression returns None.

My opinion is that each of these should be an error.

Tim Delaney