On Sat, Nov 15, 2014 at 1:29 AM, Chris Angelico email@example.com wrote:
The interaction of generators and ``StopIteration`` is currently somewhat surprising, and can conceal obscure bugs. An unexpected exception should not result in subtly altered behaviour, but should cause a noisy and easily-debugged traceback. Currently, ``StopIteration`` can be absorbed by the generator construct.
Specifically it's absorbed by the caller of the generator, because the caller doesn't know the difference between next(x) raising StopIteration because the iterator specifically wants to stop, vs because of accident.
As another alternative, how about a new iterator protocol that is defined without this ambiguity? Code at the bottom of my post to help explain: define a new method __nextx__ which doesn't use StopIteration for any signalling, instead, it returns None if there are no values to return, and returns a special value Some(v) if it wants to return a value v. Both next(it) and nextx(it) are made to work for any iterator that is defined using either protocol, but for loops and Python builtins all use nextx internally. Generators define __next__ unless you from __future__ import iterators, in which case they define __nextx__ instead.
In this way, old code can't tell the difference between accidental StopIteration and deliberate StopIteration, but new code (using nextx instead of next, and using __future__ import'd generators) can. No backwards incompatibility is introduced, and you can still insert StopIteration into a generator and get it back out -- using both next() where it is ambiguous and nextx() where it is not.
Yes, it's ugly to have two different iterator protocols, but not that ugly. In fact, this would be Python's third (I have omitted that third protocol in the below example, for the sake of clarity). I find the proposed solution more scary, in that it's sort of a "hack" to get around an old mistake, rather than a correction to that mistake, and it introduces complexity that can't be removed in principle. (Also, it's very unusual.)
class Some: def __init__(self, value): self.value = value
def next(it): v = nextx(it) if v is None: raise StopIteration return v.value
def nextx(it): if hasattr(it, '__nextx__'): v = it.__nextx__() if v is None or isinstance(v, Some): return v raise TypeError("__nextx__ must return Some(...) or None, not %r" % (v,)) if hasattr(it, '__next__'): try: return Some(it.__next__()) except StopIteration: return None raise TypeError