[Python-ideas] Using yield inside a comprehension.

[Jonathan Slenders]

Thanks Nick, nice explanation!

I was also not complaining, just wondering why.
But this makes sense, I think we should probably avoid using "yield" inside
a comprehension.


2013/11/26 Nick Coghlan <ncoghlan at gmail.com>

>
> On 27 Nov 2013 01:35, "Jonathan Slenders" <jonathan at slenders.be> wrote:
> > Futher, there is a really weird constructs possible:
> >
> > list((yield x) for x in range(10))
> > [0, None, 1, None, 2, None, 3, None, 4, None, 5, None, 6, None, 7, None,
> 8, None, 9, None]
> >
> >
> > It's a logical consequence from the translation, but I don't get the
> point.
> > Can't we create a local namespace, without wrapping it in a function?
>
> That was the original implementation I tried, and it turned out to be
> inordinately difficult to get the semantics right for lambda expressions
> that referenced iteration variables from inside the comprehension. There
> are also some ugly edge cases involving the locals() builtin that would
> need to have their semantics defined. Switching to a full lexical scope
> instead turned out to be much easier to implement while still providing
> closure semantics that matched those of generator expressions, so that's
> what I ended up implementing. This approach also resolved the "How does
> locals() work in a 3.x comprehension?" question in favour of making it work
> the same way it does in a generator expression.
>
> As others have noted, this approach of using an implicit function
> definition also results in the behaviour of yield expressions inside
> comprehensions being entirely consistent with their behaviour inside def
> statements and lambda expressions - it turns an ordinary function into a
> generator function. Is this particularly useful? Not that I've seen (while
> you can do some kinda neat one-liner hacks with it, they're basically
> incomprehensible to the reader). It's just a natural consequence of making
> comprehension semantics more consistent with those of generator expressions
> (which were already consistent with nested def statements and lambda
> expressions), and there's no compelling justification to disallow it.
>
> Regarding the PEP question, there's no dedicated PEP for the change, just
> a line item in PEP 3100 to make comprehensions more like generator
> expressions by hiding the iteration variable. There's probably a thread or
> two on the old python-3000 list about using a full function scope to do it,
> though (I seem to recall posting about it after my original pseudo-scope
> based approach failed to handle closures properly).
>
> Cheers,
> Nick.
>
> >
> >
> >
> >
> > 2013/11/26 Andrew Barnert <abarnert at yahoo.com>
> >>
> >> On Nov 26, 2013, at 6:32, Oscar Benjamin <oscar.j.benjamin at gmail.com>
> wrote:
> >>
> >> > On 26 November 2013 13:54, Jonathan Slenders <jonathan at slenders.be>
> wrote:
> >> >>
> >> >> Where do I find the PEP that describes that the following statement
> assigns
> >> >> a generator object to `values`?
> >> >
> >> > I don't think there was a PEP for this but it's a consequence of the
> >> > change to binding in list comprehensions introduced in Python 3.x
> >> > which is mentioned here:
> >> >
> http://python-history.blogspot.co.uk/2010/06/from-list-comprehensions-to-generator.html
> >> >
> >> > Essentially this:
> >> >
> >> >> values = [ (yield x) for x in range(10) ]
> >> >
> >> > Translates to the following in Python 2.x:
> >> >
> >> > _tmp = []
> >> > for x in range(10):
> >> >    _tmp.append((yield x))
> >> > values = _tmp
> >> >
> >> > However in 3.x it translates to something like:
> >> >
> >> > def _tmpfunc():
> >> >    _tmp = []
> >> >    for x in range(10):
> >> >        _tmp.append((yield x))
> >> >    return _tmp
> >> > values = _tmpfunc()
> >> >
> >> > This change was made to prevent the comprehension variable from
> >> > leaking to the enclosing scope, but as you say if the code is nested
> >> > in a function then it affects which function contains the yield
> >> > statement and the presence of a yield statement radically alters the
> >> > behaviour of a function. So in 2.x the enclosing function must become
> >> > a generator function. However in 3.x the function that is supposed to
> >> > implement the list comprehension is changed into a generator function
> >> > instead.
> >> >
> >> > $ python3
> >> > Python 3.3.2 (v3.3.2:d047928ae3f6, May 16 2013, 00:03:43) [MSC v.1600
> >> > 32 bit (Intel)] on win32
> >> > Type "help", "copyright", "credits" or "license" for more information.
> >> >>>> values = [(yield x) for x in range(10)]
> >> >>>> values
> >> > <generator object <listcomp> at 0x00E24508>
> >> >>>> def _tmpfunc():
> >> > ...     _tmp = []
> >> > ...     for x in range(10):
> >> > ...         _tmp.append((yield x))
> >> > ...     return _tmp
> >> > ...
> >> >>>> values = _tmpfunc()
> >> >>>> values
> >> > <generator object _tmpfunc at 0x00E2F7B0>
> >> >
> >> >> I assume it's equivalent to the following:
> >> >> values = (x for x in range(10))
> >> >
> >> > It will yield the same values but it will also build a list of Nones
> >> > and attach it to StopIteration:
> >>
> >> Unless you call .send on it, in which case it'll build a list of the
> values you send it and attach _that_ to StopIteration, of course.
> >>
> >> So I suppose you could use it as a coroutine version of the list
> function. Except that the number of values it takes is specified on the
> wrong end.
> >
> >
> >
> > _______________________________________________
> > Python-ideas mailing list
> > Python-ideas at python.org
> > https://mail.python.org/mailman/listinfo/python-ideas
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131127/99399532/attachment.html>