[Python-ideas] Using yield inside a comprehension.

[Jonathan Slenders]

Thanks Oscar, for the extensive explanation. The translation process is
clear to me.
But I'm not convinced that yield or yield from should be bound to this
invisible function instead of the parent.

Now that asyncio/tulip is coming, I see some use case.
Suppose that you have a list of futures, and you want to process them.
The following works, using the "gather" function.

@coroutine
def process(futures, func):
    values = yield from gather(futures)
    return [ func(v) for v in values ]

What I tried to do, with my knowledge of Python 2 generators, is the
following, but that doesn't work.

@coroutine
def process(futures, func):
    return [ func((yield from f)) for f in futures ]

They are also not equivalent. Using gather(), we have to wait before all
the futures are ready, while in the second example, we could start creating
the list on the fly. (Note that in this case "yield" instead of "yield
from" would also work.)


Futher, there is a really weird constructs possible:

list((yield x) for x in range(10))
[0, None, 1, None, 2, None, 3, None, 4, None, 5, None, 6, None, 7, None, 8,
None, 9, None]


It's a logical consequence from the translation, but I don't get the point.
Can't we create a local namespace, without wrapping it in a function?




2013/11/26 Andrew Barnert <abarnert at yahoo.com>

> On Nov 26, 2013, at 6:32, Oscar Benjamin <oscar.j.benjamin at gmail.com>
> wrote:
>
> > On 26 November 2013 13:54, Jonathan Slenders <jonathan at slenders.be>
> wrote:
> >>
> >> Where do I find the PEP that describes that the following statement
> assigns
> >> a generator object to `values`?
> >
> > I don't think there was a PEP for this but it's a consequence of the
> > change to binding in list comprehensions introduced in Python 3.x
> > which is mentioned here:
> >
> http://python-history.blogspot.co.uk/2010/06/from-list-comprehensions-to-generator.html
> >
> > Essentially this:
> >
> >> values = [ (yield x) for x in range(10) ]
> >
> > Translates to the following in Python 2.x:
> >
> > _tmp = []
> > for x in range(10):
> >    _tmp.append((yield x))
> > values = _tmp
> >
> > However in 3.x it translates to something like:
> >
> > def _tmpfunc():
> >    _tmp = []
> >    for x in range(10):
> >        _tmp.append((yield x))
> >    return _tmp
> > values = _tmpfunc()
> >
> > This change was made to prevent the comprehension variable from
> > leaking to the enclosing scope, but as you say if the code is nested
> > in a function then it affects which function contains the yield
> > statement and the presence of a yield statement radically alters the
> > behaviour of a function. So in 2.x the enclosing function must become
> > a generator function. However in 3.x the function that is supposed to
> > implement the list comprehension is changed into a generator function
> > instead.
> >
> > $ python3
> > Python 3.3.2 (v3.3.2:d047928ae3f6, May 16 2013, 00:03:43) [MSC v.1600
> > 32 bit (Intel)] on win32
> > Type "help", "copyright", "credits" or "license" for more information.
> >>>> values = [(yield x) for x in range(10)]
> >>>> values
> > <generator object <listcomp> at 0x00E24508>
> >>>> def _tmpfunc():
> > ...     _tmp = []
> > ...     for x in range(10):
> > ...         _tmp.append((yield x))
> > ...     return _tmp
> > ...
> >>>> values = _tmpfunc()
> >>>> values
> > <generator object _tmpfunc at 0x00E2F7B0>
> >
> >> I assume it's equivalent to the following:
> >> values = (x for x in range(10))
> >
> > It will yield the same values but it will also build a list of Nones
> > and attach it to StopIteration:
>
> Unless you call .send on it, in which case it'll build a list of the
> values you send it and attach _that_ to StopIteration, of course.
>
> So I suppose you could use it as a coroutine version of the list function.
> Except that the number of values it takes is specified on the wrong end.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131126/acfaa51b/attachment-0001.html>