(Note: this is an off-topic side thread, unrelated to assignment expressions. Inline comment below.)

On Fri, May 11, 2018 at 9:08 AM, Chris Angelico <rosuav@gmail.com> wrote:
On Fri, May 11, 2018 at 9:15 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
> * *maybe* discover that even the above expansion isn't quite accurate, and
> that the underlying semantic equivalent is actually this (one way to
> discover this by accident is to have a name error in the outermost iterable
> expression):
>
>     def _genexp(_outermost_iter):
>         for x in _outermost_iter:
>             yield x
>
>     _result = _genexp(_outermost_iter)
>
> * and then realise that the optimised list comprehension form is essentially
> this:
>
>     def _listcomp(_outermost_iter):
>         result = []
>         for x in _outermost_iter:
>             result.append(x)
>         return result
>
>     _result = _listcomp(data)
>
> Now that "yield" in comprehensions has been prohibited, you've learned all
> the edge cases at that point

Not quite! You missed one, just because comprehensions aren't weird
enough yet. AFAIK you can't tell with the list comp, but with the
genexp you can (by not iterating over it).

>     def _genexp(_outermost_iter):
>         for x in _outermost_iter:
>             yield x
>
>     _result = _genexp(data)

It's actually this:

     def _genexp(_outermost_iter):
         for x in _outermost_iter:
             yield x

     _result = _genexp(iter(_outermost_iter))

I don't think there's anything in the main documentation that actually
says this, although PEP 289 mentions it in the detaily bits. [1]

ChrisA

[1] https://www.python.org/dev/peps/pep-0289/#the-details

I'm not sure this is the whole story. I tried to figure out how often __iter__ is called in a genexpr. I found that indeed I see iter() is called as soon as the generator is brought to life, but it is *not* called a second time the first time you call next(). However the translation you show has a 'for' loop which is supposed to call iter() again. So how is this done? It seems the generated bytecode isn't equivalent to a for-loop, it's equivalent to s while loop that just calls next().

Disassembly of a regular generator:

def foo(a):
  for x in a: yield x

  2           0 SETUP_LOOP              18 (to 20)
              2 LOAD_FAST                0 (a)
              4 GET_ITER
        >>    6 FOR_ITER                10 (to 18)
              8 STORE_FAST               1 (x)
             10 LOAD_FAST                1 (x)
             12 YIELD_VALUE
             14 POP_TOP
             16 JUMP_ABSOLUTE            6
        >>   18 POP_BLOCK
        >>   20 LOAD_CONST               0 (None)
             22 RETURN_VALUE


But for a generator:

g = (x for x in C())

  1           0 LOAD_FAST                0 (.0)
        >>    2 FOR_ITER                10 (to 14)
              4 STORE_FAST               1 (x)
              6 LOAD_FAST                1 (x)
              8 YIELD_VALUE
             10 POP_TOP
             12 JUMP_ABSOLUTE            2
        >>   14 LOAD_CONST               0 (None)
             16 RETURN_VALUE

Note the lack of SETUP_LOOP and GET_ITER (but otherwise they are identical).

--
--Guido van Rossum (python.org/~guido)