[Python-ideas] Generator syntax hooks?

[Paul Moore]

On 10 August 2017 at 14:42, Steven D'Aprano <steve at pearwood.info> wrote:
> I don't think it is confusing. Regardless of the implementation, the
> meaning of:
>
> [expression for x in sequence while condition]
>
> should (I believe) be obvious to anyone who already groks comprehension
> syntax. The mapping to a for-loop is admittedly a tad more complex:
>
> result = []
> for x in sequence:
>     if not condition: break
>     result.append(expression)
>
> but I'm yet to meet anyone who routinely and regularly reads
> comprehensions by converting them to for loops like that. And if they
> did, all they need do is mentally map "while condition" to "if not
> condition: break" and it should all Just Work™.

The hard part is the interaction between if and while.

Consider (expr for var in seq if cond1 while cond2):

This means:

for var in seq:
    if cond1:
        if not cond2: break
        yield expr

Note that unlike all other comprehension clauses (for and if) while
doesn't introduce a new level of nesting. That's an inconsistency, and
while it's minor, it would need clarifying (my original draft of this
email was a mess, because I misinterpreted how if and while would
interact, precisely over this point). Also, there's a potential issue
here - consider

    [expr for var in even_numbers() if is_odd(var) while var < 100]

This is an infinite loop, even though it has a finite termination
condition (var < 100), because we only test the termination condition
if var is odd, which it never will be.

Obviously, this is a contrived example. And certainly "don't do that,
then" is a valid response. But my instinct is that people are going to
get this wrong - *especially* in a maintenance environment. That
example could have started off being "for var in count(0)" and then
someone realised they could "optimise" it by omitting odd numbers,
introducing the bug in the process. (And I'm sure real life code could
come up with much subtler examples ;-))

Overall, I agree with Steven's point. It seems pretty obvious what the
intention is, and while it's probably possible to construct examples
that are somewhat unclear,

1. The mechanical rule gives an explicit meaning
2. People shouldn't be writing such complex comprehensions, so if the
rule doesn't give what they expect, they can always rewrite the code
with an explicit (and clearer) loop.

But while I think this says that the above interpretation of while is
the only sensible one, and in general other approaches are unlikely to
be as natural, I *don't* think that it unequivocally says that
allowing while is a good thing. It may still be better to omit it, and
force people to state their intent explicitly (albeit a bit more
verbosely).

Paul