[Python-ideas] free variables in generator expressions

[Greg Ewing]

Arnaud Delobelle wrote:

> The following is about how to treat 'free names' inside generator
> expressions.  I argue that these names should be bound to their values
> when the generator expression is created,

My opinion is that this is the wrong place to attack the
problem. Note that it's not only generator expressions that
have this issue -- the same thing can trip you up with
lambdas as well.

It's instructive to consider why other languages with
lexical scoping and first-class functions don't seem to
have this problem to the same extent. In Scheme, for
example, the reason is that its looping constructs
usually create a new binding for the loop variable
on each iteration, instead of re-using the same one.
So if you return a lambda from inside the loop, each
one lives in a different lexical environment and
sees a different value for the loop variable.

If Python's for-loop did the same thing, I suspect that
this problem would turn up much less frequently.

In CPython, there's a straightforward way to implement
this: if the variable is used in an inner function, and
is therefore in a cell, create a new cell each time
round the loop instead of replacing the contents of the
existing one. (If the variable isn't in a cell, there's
no need to change anything.)

Note that this wouldn't interfere with using the loop
variable after the loop has finished -- the variable
is still visible to the whole function, and the following
code just sees whatever is in the last created cell.

--
Greg