[Python-ideas] Change how Generator Expressions handle StopIteration

[Steven D'Aprano]

On Thu, Nov 13, 2014 at 12:04:52PM -0800, Guido van Rossum wrote:

> Do we need a PEP for this or can we just go ahead? I haven't heard any
> pushback on the basic premise that this is a problem we'd like to fix.

*puts hand up*

I'm not convinced that this is *a problem to be fixed*. At worst it is a 
"gotcha" to be aware of.

The current behaviour is simple to understand: raising StopIteration 
halts the generator, end of story. I'm still not sure that I understand 
what the proposed fix is (a PEP will be good to explain that), but if I 
have understood it correctly, it turns a simple concept like 
"StopIteration halts the generator" into something more complicated: 
some StopIterations will halt the generator, others will be chained to a 
new, unrelated exception.

Judging by the complete lack of questions about this on the tutor and 
python-list mailing lists, the slight difference in behaviour between 
generator expressions and comprehensions is not an issue in practice. 
I've seen people ask about the leakage of variables from comprehensions; 
I've never seen people ask about the different treatment of 
StopIteration.

I have, however, seen people *rely* on that different treatment. E.g. to 
implement a short-circuiting generator expression that exits when a 
condition is reached:

    (expr for x in sequence if cond or stop())

where stop() raises StopIteration and halts the generator. If this 
change goes ahead, it will break code that does this.

Having generator expressions and comprehensions behave exactly the same 
leads to the question, why do we have comprehensions? I guess the answer 
is "historical reasons", but from a pragmatic point of view being able 
to choose between

    [expr for x in sequence]
    list(expr for x in sequence)

depending on how you want StopIteration to be treated may be useful.


-- 
Steven