[Python-Dev] Tricky way of of creating a generator via a comprehension expression

[Nick Coghlan]

On 23 November 2017 at 23:04, Ivan Levkivskyi <levkivskyi at gmail.com> wrote:

> I don't see why this particular case qualifies for such a radical measure
> as an exception to syntactic rules,
> instead of just fixing it (sorry Nick :-)
>

I've posted in more detail about this to the issue tracker, but the
argument here is: because making it behave differently from the way it does
now while still hiding the loop iteration variable potentially requires
even more radical revisions to the lexical scoping rules :)

If somebody can come up with a clever trick to allow yield inside a
comprehension to jump levels in a relatively intuitive way, that would
actually be genuinely cool, but the lexical scoping rules mean it's
trickier than it sounds.

Now that I frame the question that way, though, I'm also remembering that
we didn't have "yield from" yet when I wrote the current comprehension
implementation, and given that, it may be as simple as having an explicit
yield expression in a comprehension imply delegation to a subgenerator.

If we went down that path, then a list comprehension like the following:

    results = [(yield future) for future in list_of_futures]

might be compiled as being equivalent to:

    def __listcomp_generator(iterable):
        result = []
        for future in iterable:
            result.append((yield future))
        return result

    results = yield from _listcomp_generator(list_of_futures)

The only difference between the current comprehension code and this idea is
"an explicit yield expression in a comprehension implies the use of 'yield
from' when calling the nested function".

For generator expressions, the adjustment would need to be slightly
different: for those, we'd either need to prohibit yield expressions, or
else say that if there's an explicit yield expression present anywhere,
then we drop the otherwise implied yield expression.

If we went down the latter path, then:

    gen = ((yield future) for future in list_of_futures)

and:

    gen = (future for future in list_of_futures)

would be two different ways of writing the same thing.

The pay-off for allowing it would be that you could write things like:

    gen = (f(yield future) for future in list_of_futures)

as a shorthand equivalent to:

    def gen(list_of_futures=list_of_futures):
        for future in list_of_futures:
            f(yield future)

(Right now, you instead get "yield f(yield future)" as the innermost
statement, which probably isn't what you wanted)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20171124/18e84150/attachment-0001.html>