[Python-Dev] PEP 479: Change StopIteration handling inside generators

[Ron Adam]

On 11/22/2014 02:16 PM, Chris Angelico wrote:
> On Sun, Nov 23, 2014 at 6:49 AM, Ron Adam<ron3200 at gmail.com>  wrote:
>> >
>> >OPTION 1:
>> >
>> >Make comprehensions act more like generator expressions.
>> >
>> >It would mean a while loop in the object creation point is converted to a
>> >for loop. (or something equivalent.)
>> >
>> >Then both a comprehension and a generator expressions can be viewed as
>> >defining iterators, with the same behaviour, rather than comprehensions
>> >defining the body of the loop, which has the different but valid behaviour
>> >of StopIteration escaping.
>> >
>> >This would make explaining them a*lot*  easier as they become the same thing
>> >used in a different context, rather than two different things used in what
>> >appears to be similar contexts.
>> >
>> >
>> >I think this fits with what Guido wants, but does so in a narrower scope,
>> >only effecting comprehensions.  StopIteration is less likely to leak out.
> A list comp is usually compared to a statement-form loop.
>
> def stop(): raise StopIteration
> lst = [stop() for x in (1,2,3)]

> lst = []
> for x in (1,2,3):
>      lst.append(stop())
>
> At the moment, these are identical (virtually; the pedantic will point
> out that 'x' doesn't leak out of the comprehension) - in each case,
> the exception raised by the body will bubble up and be printed to the
> console.

This is what I meant by leaking out.  A StopIteration bubbles up.  And your 
examples match my understanding. :-)




> But a generator expression is different:

Yes, but they work as I expect them to.



> lst = list(stop() for x in (1,2,3))
>
> In this form, lst is an empty list, and nothing is printed to the
> console.

I think that is what it should do.  I think of it this way...

 >>> def stop(): raise StopIteration
...
 >>> def ge():
...    for value in (stop() for x in (1,2,3)):
...       yield value
...
 >>> list(ge())
[]

Notice the entire body of the comprehension is in the for loop header, and 
no parts of it are in the body except the reference to the already assigned 
value.

The StopIteration is caught by the outer for loop.  Not the for loop in the 
generator expression, or iterator part.


> Making comprehensions work more like generator expressions
> would, IMO, imply making the same change to all for loops: having a
> StopIteration raised by the body of the loop quietly terminate the
> loop.

I'm not suggesting making any changes to generator expressions or for loops 
at all.  They would continue to work like they currently do.


Cheers,
    Ron


> This is backwards. Most of Python is about catching exceptions as
> narrowly as possible: you make your "try" blocks small, you use
> specific exception types rather than bare "except:" clauses, etc, etc,
> etc. You do your best to catch ONLY those exceptions that you truly
> understand, unless you're writing a "catch, log, and reraise" or
> "catch, log, and go back for more work" generic handler. A 'for' loop
> currently is written more-or-less like this:
>
> for var in expr:
>      body
>
> it = iter(expr)
> while True:
>      try: var = next(it)
>      except StopIteration: break
>      body
>
> But this suggestion of yours would make it become:
> it = iter(expr)
> while True:
>      try:
>          var = next(it)
>          body
>      except StopIteration: break

> I believe this to be the wrong direction to make the change. Instead,
> generator expressions should be the ones to change, because that's a
> narrowing of exception-catching scope. Currently, every generator
> function is implicitly guarded by "try: ...... except StopIteration:
> pass". This is allowing errors to pass silently, to allow a hack
> involving non-local control flow (letting a chained function call
> terminate a generator) - violating the Zen of Python.
>
> ChrisA