[Python-Dev] Tricky way of of creating a generator via a comprehension expression

[Ivan Levkivskyi]

On 22 November 2017 at 18:15, Paul Moore <p.f.moore at gmail.com> wrote:

> On 22 November 2017 at 16:47, Ivan Levkivskyi <levkivskyi at gmail.com>
> wrote:
> > On 22 November 2017 at 17:43, Paul Moore <p.f.moore at gmail.com> wrote:
> >>
> >> On 22 November 2017 at 16:30, Ivan Levkivskyi <levkivskyi at gmail.com>
> >> wrote:
> >> > On 22 November 2017 at 17:24, Antoine Pitrou <solipsis at pitrou.net>
> >> > wrote:
> >> >> Given a comprehension (e.g. list comprehension) is expected to work
> >> >> nominally as `constructor(generator expression)`
> >> >
> >> > As Yury just explained, these two are not equivalent if there is an
> >> > `await`
> >> > in the comprehension/generator expression.
> >>
> >> As Antoine said, people *expect* them to work the same.
> >
> >
> > The difference is that a generator expression can be used independently,
> one
> > can assign it to a variable etc. not necessary to wrap it into a list()
> > Anyway, can you propose an equivalent "defining" code for both?
> Otherwise it
> > is not clear what you are defending.
>
> [...]
>
> 1. List comprehensions expand into nested for/if statements in the
> "obvious" way - with an empty list created to start and append used to
> add items to it.
>    1a. Variables introduced in the comprehension don't "leak" (see below).
> 2. Generator expressions expand into generators with the same "nested
> loop" behaviour, and a yield of the generated value.
> 3. List comprehensions are the same as list(the equivalent generator
> expression).
>
>
Paul, OK, I think how to formulate these rules more "precisely" so that
they will be all consistent even if there is a `yield` inside.
The key idea is that neither comprehensions nor generator expressions
should create a function scope surrounding the `expr` in (expr for ind in
iterable) and [expr for ind in iterable].
(this still can be some hidden implementation detail)

So as Serhiy proposed in one of his first posts any comprehensions and
generator expressions with `yield` are not valid outside functions.
If such comprehension or generator expression is inside a function, then it
makes it a generator, and all the `yiled`s are yielded from that generator,
for example:

def fun_gen():
    return list((yield i) for i in range(3))

should work as following:

g = func_gen()

g.send(42)
0
g.send(43)
1
g.send(44)
2
try:
    g.send(45)
except StopIteration as e:
    assert e.value  == [42, 43, 44]

And exactly the same with

def fun_comp():
    [(yield i) for i in range(3)]

I hope with this we can close the no-async part of the problem.
Currently this is not how it works, and should be fixed. Do you agree?

The async part can then be considered separately.

--
Ivan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20171122/d567145a/attachment-0001.html>