On Sat, Nov 22, 2014 at 10:06 AM, Chris Barker email@example.com wrote:
I think the point of this PEP is that the author og a generator function is thinking about using "yield" to provide the next value, and return (explicit or implicit) to stop the generation of objects. That return is raise a StopIteration, but the author isn't thinking about it.
So why would they want to think about having to trap StopIteration when calling other functions.
While the author of a iterator class is thinking about the __next__ method and raising a StopIteration to terminate. So s/he would naturally think about trapping StopIteration when calling functions?
I suppose that makes some sense, but to me it seems like a generator function is a different syntax for creating what is essentially the same thing -- why shouldn't it have the same behavior?
Let's suppose you use a Python-like macro language to generate Python code. In the macro language, you can write stuff like this:
class X: iterator: return 5 return 6 return 7 iterator_done
And it will get compiled into something like this:
class X: def __init__(self): self._iter_state = 0 def __iter__(self): return self def __next__(self): self._iter_state += 1 if self._iter_state == 1: return 5 if self._iter_state == 2: return 6 if self._iter_state == 3: return 7 raise StopIteration
This is a reasonably plausible macro language, right? It's basically still a class definition, but it lets you leave out a whole bunch of boilerplate. Now, the question is: As you write the simplified version, should you ever need to concern yourself with StopIteration? I posit no, you should not; it's not a part of the macro language at all. Of course, if this *were* how things were done, it would probably be implemented as a very thin wrapper, exposing all its details to your code; but there's no reason that it *needs* to be so thin. The language you're writing in doesn't need to have any concept of a StopIteration exception, because it doesn't need to use an exception to signal "no more results".
and of you are writing a generator, presumably you know how it's going to get use -- i.e. by somethign that expects a StopIteration -- it's not like you're ignorant of the whole idea.
Not necessarily. Can we get someone here who knows asyncio and coroutines, and can comment on the use of such generators?
Consider this far fetched situation:
Either a iterator class or a generator function could take a function object to call to do part of its work. If that function happened to raise a StopIteration -- now the user would have to know which type of object they were workign with, so they would know how to handle the termination of the iter/gener-artion
Either a __getattr__ or a __getitem__ could use a helper function to do part of its work, too, but either the helper needs to know which, or it needs some other way of signalling. They're different protocols, so they're handled differently. If Python wanted to conflate all of these, there could be a single "NoReturnValue" exception, used by every function which needs to be able to return absolutely any object and also to be able to signal "I don't have anything to return". But no, Python has separate exceptions for signalling "I don't have any such key", "I don't have any such attribute", and "I don't have any more things to iterate over". Generators don't need any of them, because - like my macro language above - they have two different keywords and two different byte-codes (yield vs return).
In many cases, the helper function doesn't actually need the capability to return *absolutely any object*. In that case, the obvious solution would be to have it return None to say "nothing to return", and then the calling function can either translate that into the appropriate exception, or return rather than yielding, as appropriate. That would also make the helper more useful to other stand-alone functions. But even if your helper has to be able to return absolutely anything, you still have a few options:
1) Design the helper as part of __next__, and explicitly catch the exception.
def nexthelper(): if condition: return value raise StopIteration
def __next__(self): return nexthelper()
def gen(): try: yield nexthelper() except StopIteration: pass
2) Write the helper as a generator, and explicitly next() it if you need that:
def genhelper(): if condition: yield value
def __next__(self): return next(genhelper())
def gen(): yield from genhelper()
3) Return status and value. I don't like this, but it does work.
def tuplehelper(): if condition: return True, value return False, None
def __next__(self): ok, val = tuplehelper() if ok: return val raise StopIteration
def gen(): ok, val = tuplehelper() if ok: yield val
All these methods work perfectly, because they have a clear boundary between protocols. If you want to write a __getitem__ that calls on the same helper, you can do that, and have __getitem__ itself raise appropriately if there's nothing to return.
AFAIU, the current distinction between generators and iterators is how they are written -- i.e. syntax, essentially. But this PEP would change the behavior of generators in some small way, creating a distinction that doesn't currently exist.
Generators are currently a leaky abstraction for iterator classes. This PEP plugs a leak that's capable of masking bugs.
I guess this is where I'm not sure -- it seems to me that the behavior of generators is being change, not the syntax -- so while mostly of concern to generator authors, it is, in fact, a chance in behavior that can be seen by the consumer of (maybe only an oddly designed) generator. In practice, that difference may only matter to folks using that particular hack in generator expression, but it is indeed a change.
The only way a consumer will see a change of behaviour is if the generator author used this specific hack (in which case, instead of the generator quietly terminating, a RuntimeError will bubble up - hopefully all the way up until a human sees it). In terms of this PEP, that's a bug in the generator. Bug-free generators will not appear any different to the consumer.