[Python-Dev] Tricky way of of creating a generator via a comprehension expression

[Paul Moore]

On 22 November 2017 at 13:53, Ivan Levkivskyi <levkivskyi at gmail.com> wrote:
> On 22 November 2017 at 14:38, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>
>> On Wed, 22 Nov 2017 15:03:09 +0200
>> Serhiy Storchaka <storchaka at gmail.com> wrote:
>> >  From
>> >
>> > https://stackoverflow.com/questions/45190729/differences-between-generator-comprehension-expressions.
>> >
>> >      g = [(yield i) for i in range(3)]
>> >
>> > Syntactically this looks like a list comprehension, and g should be a
>> > list, right? But actually it is a generator. This code is equivalent to
>> > the following code:
>> >
>> >      def _make_list(it):
>> >          result = []
>> >          for i in it:
>> >              result.append(yield i)
>> >          return result
>> >      g = _make_list(iter(range(3)))
>> >
>> > Due to "yield" in the expression _make_list() is not a function
>> > returning a list, but a generator function returning a generator.
>> >
>> > This change in semantic looks unintentional to me. It looks like leaking
>> > an implementation detail.
>>
>> Perhaps we can deprecate the use of "yield" in comprehensions and make
>> it a syntax error in a couple versions?
>>
>> I don't see a reason for writing such code rather than the more
>> explicit variants.  It looks really obscure, regardless of the actual
>> semantics.
>
>
> People actually try this (probably simply because they like comprehensions)
> see two mentioned Stackoverflow questions, plus there are two b.p.o. issues.
> So this will be a breaking change. Second, recent PEP 530 allowed writing a
> similar comprehensions with `await`:
>
>     async def process(funcs):
>         result = [await fun() for fun in funcs]  # OK
>         ...
>
> Moreover, it has the semantics very similar to the proposed by Serhiy for
> `yield` (i.e. equivalent to for-loop without name leaking into outer scope).
> Taking into account that the actual fix is not so hard, I don't think it
> makes sense to have all the hassles of deprecation period.

I agree with Antoine. This (yield in a comprehension) seems far too
tricky, and I'd rather it just be rejected. If I saw it in a code
review, I'd certainly insist it be rewritten as an explicit loop.

Paul