[Python-ideas] Using yield inside a comprehension.

[Oscar Benjamin]

On 26 November 2013 13:54, Jonathan Slenders <jonathan at slenders.be> wrote:
>
> Where do I find the PEP that describes that the following statement assigns
> a generator object to `values`?

I don't think there was a PEP for this but it's a consequence of the
change to binding in list comprehensions introduced in Python 3.x
which is mentioned here:
http://python-history.blogspot.co.uk/2010/06/from-list-comprehensions-to-generator.html

Essentially this:

   > values = [ (yield x) for x in range(10) ]

Translates to the following in Python 2.x:

_tmp = []
for x in range(10):
    _tmp.append((yield x))
values = _tmp

However in 3.x it translates to something like:

def _tmpfunc():
    _tmp = []
    for x in range(10):
        _tmp.append((yield x))
    return _tmp
values = _tmpfunc()

This change was made to prevent the comprehension variable from
leaking to the enclosing scope, but as you say if the code is nested
in a function then it affects which function contains the yield
statement and the presence of a yield statement radically alters the
behaviour of a function. So in 2.x the enclosing function must become
a generator function. However in 3.x the function that is supposed to
implement the list comprehension is changed into a generator function
instead.

$ python3
Python 3.3.2 (v3.3.2:d047928ae3f6, May 16 2013, 00:03:43) [MSC v.1600
32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> values = [(yield x) for x in range(10)]
>>> values
<generator object <listcomp> at 0x00E24508>
>>> def _tmpfunc():
...     _tmp = []
...     for x in range(10):
...         _tmp.append((yield x))
...     return _tmp
...
>>> values = _tmpfunc()
>>> values
<generator object _tmpfunc at 0x00E2F7B0>

> I assume it's equivalent to the following:
> values = (x for x in range(10))

It will yield the same values but it will also build a list of Nones
and attach it to StopIteration:

>>> values = [(yield x) for x in range(3)]
>>> values
<generator object <listcomp> at 0x00E1CCD8>
>>> next(values)
0
>>> next(values)
1
>>> next(values)
2
>>> next(values)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration: [None, None, None]

> The reason for asking this is that I see no point in using the first syntax
> and that the first has another meaning in Python 2.7, if used inside a
> function.

This has been mentioned before and AFAICT it is an unintended artefact
of the new definitions for comprehensions and yield and the fact that
'return _tmp' works inside a generator function. I haven't seen a use
for this behaviour yet.


Oscar