[Python-Dev] Tricky way of of creating a generator via a comprehension expression

[Hrvoje Niksic]

Guido van Rossum writes:
>         And my mind boggles when considering a generator expression
>         containing yield that is returned from a function. I tried this
>         and cannot say I expected the outcome:
> 
>             def f():
>                 return ((yield i) for i in range(3))
>             print(list(f()))
> 
>         In both Python 2 and Python 3 this prints
> 
>             [0, None, 1, None, 2, None]
> 
>         Even if there's a totally logical explanation for that, I still
>         don't like it, and I think yield in a comprehension should be
>         banned. From this it follows that we should also simply ban
>         yield  from comprehensions.
> 

Serhiy Storchaka writes:
>     This behavior doesn't look correct to me and Ivan.
The behavior is surprising, but it seems quite consistent with how 
generator expressions are defined in the language. A generator 
expression is defined by the language reference as "compact generator 
notation in parentheses", which yields (sic!) a "new generator object".

I take that to mean that a generator expression is equivalent to 
defining and calling a generator function. f() can be transformed to:

def f():
     def _gen():
         for i in range(3):
             ret = yield i
             yield ret
     return _gen()

The transformed version shows that there are *two* yields per iteration 
(one explicitly written and one inserted by the transformation), which 
is the reason why 6 values are produced. The None values come from list 
constructor calling __next__() on the generator, which (as per 
documentation) sends None into the generator. This None value is yielded 
after the "i" is yielded, which is why Nones follow the numbers.

Hrvoje