[Python-Dev] Tricky way of of creating a generator via a comprehension expression

[Serhiy Storchaka]

23.11.17 18:08, Guido van Rossum пише:
> This thread is still going over the speed limit. Don't commit anything 
> without my explicit approval.

I'm not going to write a single line of code while the decision about 
this issue is not made. This is not easy issue.

> A problem with dropping the "function-ness" of the comprehension by 
> renaming the variables (as Ivan/Serhiy's plan seems to be?) would be 
> what does it look like in the debugger -- can I still step through the 
> loop and print the values of expressions? Or do I have to know the 
> "renamed" names?

Does the debugger supports stepping inside a comprehension? Isn't the 
whole comprehension a single logical line? It this is supported the the 
debugger should see a local variable with a strange name ".0". This is a 
leaked implementation detail. It wouldn't be surprised if details of 
other implementation will be leaked too in the debugger. The debugger 
can unmangle the inner variable names and hide local variables in the 
outer scope.

> And my mind boggles when considering a generator expression containing 
> yield that is returned from a function. I tried this and cannot say I 
> expected the outcome:
> 
>    def f():
>        return ((yield i) for i in range(3))
>    print(list(f()))
> 
> In both Python 2 and Python 3 this prints
> 
>    [0, None, 1, None, 2, None]
> 
> Even if there's a totally logical explanation for that, I still don't 
> like it, and I think yield in a comprehension should be banned. From 
> this it follows that we should also simply ban yield  from comprehensions.

This behavior doesn't look correct to me and Ivan. Ivan explained that 
this function should be rough equivalent to

    def f():
        t = [(yield i) for i in range(3)]
        return (x for x in t)

which should be equivalent to

    def f():
        t = []
        for i in range(3):
           t.append((yield i))
        return (x for x in t)

and list(f()) should be [1, 2, 3].

But while I know how to fix yield in comprehensions (just inline the 
code with some additions), I have no ideas how to fix yield in 
generators. Me and Ivan agreed that SyntaxError in this case is better 
than the current behavior. Maybe we will find how to implement the 
expected behavior.