On 25.01.2017 07:28, Joe Jevnik wrote:

That was a long way to explain what the problem was. I think that that solution is to stop using `yield` in comprehensions because it is confusing, or to make `yield` in a comprehension a syntax error.

Same here; mixing comprehensions and yield (from) can't be explained easily. A SyntaxError would be most appropriate. Regards, Sven

On Jan 25, 2017 8:16 AM, "Joe Jevnik" wrote: List, set, and dict comprehensions compile like: # input result = [expression for lhs in iterator_expression] # output def comprehension(iterator): out = [] for lhs in iterator: out.append(expression) return out result = comprehension(iter(iterator_expression)) When you make `expression` a `yield` the compiler thinks that `comprehension` is a generator function instead of a normal function. We can manually translate the following comprehension: result = [(yield n) for n in (0, 1)] def comprehension(iterator): out = [] for n in iterator: # (yield n) as an expression is the value sent into this generator out.append((yield n)) return out result = comprehension(iter((0, 1))) We can see this in the behavior of `send` on the resulting generator: In [1]: g = [(yield n) for n in (0, 1)] In [2]: next(g) Out[2]: 0 In [3]: g.send('hello') Out[3]: 1 In [4]: g.send('world') --------------------------------------------------------------------------- StopIteration Traceback (most recent call last) <ipython-input-4-82c589e0c27d> in <module>() ----> 1 g.send('world') StopIteration: ['hello', 'world'] The `return out` gets translated into `raise StopIteration(out)` because the code is a generator. The bytecode for this looks like: In [5]: %%dis ...: [(yield n) for n in (0, 1)] ...: <module> -------- 1 0 LOAD_CONST 0 () 3 LOAD_CONST 1 ('<listcomp>') 6 MAKE_FUNCTION 0 9 LOAD_CONST 5 ((0, 1)) 12 GET_ITER 13 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 16 POP_TOP 17 LOAD_CONST 4 (None) 20 RETURN_VALUE <module>.<listcomp> ------------------- 1 0 BUILD_LIST 0 3 LOAD_FAST 0 (.0) >> 6 FOR_ITER 13 (to 22) 9 STORE_FAST 1 (n) 12 LOAD_FAST 1 (n) 15 YIELD_VALUE 16 LIST_APPEND 2 19 JUMP_ABSOLUTE 6 >> 22 RETURN_VALUE In `<module>` you can see us create the <listcomp> function, call `iter` on `(0, 1)`, and then call `<listcomp>(iter(0, 1))`. In `<listcomp>` you can see the loop like we had above, but unlike a normal comprehension we have a `YIELD_VALUE` (from the `(yield n)`) in the comprehension. The reason that this is different in Python 2 is that list comprehension, and only list comprehensions, compile inline. Instead of creating a new function for the loop, it is done in the scope of the comprehension. For example: # input result = [expression for lhs in iterator_expression] # output result = [] for lhs in iterator_expression: result.append(lhs) This is why `lhs` bleeds out of the comprehension. In Python 2, adding making `expression` a `yield` causes the _calling_ function to become a generator: def f(): [(yield n) for n in range(3)] # no return # py2