[Python-Dev] [issue6673] Py3.1 hangs in coroutine and eats up all memory

[Stefan Behnel]

[moving this from the bug tracker]

Alexandre Vassalotti wrote:
> Alexandre Vassalotti added the comment:
> 
> Not a bug.
> 
> The list comprehension in your chunker:
> 
>     while True:
>         target.send([ (yield) for i in range(chunk_size) ])
> 
> is equivalent to the following generator in Python 3:
> 
>     while True:
>         def g():
>             for i in range(chunk_size):
>                 yield (yield)
>         target.send(list(g()))
>
> This clearly needs not what you want.

Does this do anything meaningful, or would it make sense to output a
compiler warning (or better: an error) here?

Using yield in a comprehension (as opposed to a generator expression, which
I intuitively expected not to work) doesn't look any dangerous at first
glance, so it was quite surprising to see it fail that drastically.

This is also an important issue for other Python implementations. Cython
simply transforms comprehensions into the equivalent for-loop, so when we
implement PEP 342 in Cython, we will have to find a way to emulate
CPython's behaviour here (unless we decide to stick with Py2.x sematics,
which would not be my preferred solution).

Stefan


> So, just rewrite your code using for-loop:
> 
>     while True:
>         result = []
>         for i in range(chunk_size):
>             result.append((yield))
>         target.send(result)
> 
> ----------
> nosy: +alexandre.vassalotti
> resolution:  -> invalid
> status: open -> closed