On Sat, Jan 23, 2016 at 3:53 AM, Stephen J. Turnbull <stephen@xemacs.org> wrote:

Andrew Barnert via Python-ideas writes:

> powers = [lambda x: x**i for i in range(10)]

> This gives you ten functions that all return x**9, which is
> probably not what you wanted.

> The reason this is a problem is that Python uses "late binding",
> which in this context means that each of those functions is a
> closure that captures the variable i in a way that looks up the
> value of i at call time. All ten functions capture the same
> variable, and when you later call them, that variable's value is
> 9.

Actually it doesn't look up the value at call time, but each time it's used. This technicality matters if in between uses you call something that has write access to the same variable (typically using nonlocal) and modifies it.

But this explanation going to confuse people who understand the
concept of variable in Python to mean names that are bound and
re-bound to objects. The comprehension's binding of i disappears
before any element of powers can be called. So from their point of
view, either that expression is an error, or powers[i] closes over a
new binding of the name "i", specific to "the lambda's scope" (see
below), to the current value of i in the comprehension.

But this seems to refer to a very specific definition of "binding" that doesn't have root in Python's semantic model. I suppose it may come from Lisp (which didn't influence Python quite as much as people think :-).

So I think what you're saying here comes down that it will confuse people who misunderstand Python's variables. Given that the misunderstanding you're supposing here is pretty specific (it's not just due to people who've never thought much about variables) I'm not sure I care much.

Of course the same phenomenon is observable with other scopes. In
particular global scope behaves this way, as importing this file

i = 0
def f(x):
return x + i
i = 1

and calling f(0) will demonstrate. But changing the value of a
global, used the way i is here, within a library module is a rather
unusual thing to do; I doubt people will observe it.

I disagree again: in interactive mode most of what you do is global and you will see this quite often.

And all scopes in Python behave the same way.

Also, once again the semantics of lambda (specifically, that unlike
def it doesn't create a scope)

Uh, what? I can sort of guess what you are referring to here (namely, that no syntactic construct permissible in a lambda can assign to a local variable -- or any variable, for that matter) but it certainly has a scope (to hold the arguments, which are just variables, as one quickly learns from experimenting with the arguments to a function defined using def).

seem to be a source of confusion more
than anything else. Maybe it's possible to exhibit the same issue
with def, but the def equivalent to the above lambda

>>> def make_increment(i):
... def _(x):
... return x + i
... return _
...
>>> funcs = [make_increment(j) for j in range(3)]
>>> [f(0) for f in funcs]
[0, 1, 2]

closes over i in the expected way. (Of course in practicality, it's
way more verbose, and in purity, it's not truly equivalent since
there's at least one extra nesting of scope involved.)

It's such a strawman that I'm surprised you bring it up. Who would even *think* of using that idiom as equivalent to the simple lambda?

If I were to deconstruct the original statement, I would start by replacing the list comprehension with a plain old for loop. That would also not be truly equivalent because the comprehension introduces a scope while the for loop doesn't, but the difference only matters if it stomps on another variable -- the semantics relative to the lambda are exactly the same. In particular, this example exhibits the same phenomenon without using a comprehension:

powers = []

for i in range(10):

powers.append(lambda x: x**i)

This in turn can be rewritten without changing the semantics related to scopes using a def that's equivalent (truly equivalent except for its __name__ attribute!):

powers = []

for i in range(10):

def f(x):
return x**i

powers.append(f)

(Note that the leakage of f here is irrelevant to the problem.)

This has the same problem, without being distracted by lambda or comprehensions, and we can now explore its semantics through experimentation. We could even unroll the for loop and get the same issue:

powers = []

i = 0

def f(x):
return x**i

powers.append(f)

i = 1

def f(x):
return x**i

powers.append(f)

# Etc.

While

>>> def make_increment():
... def _(x):
... return x + i
... return _
...
>>> funcs = [make_increment() for i in range(3)]
>>> [f(0) for f in funcs]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 1, in <listcomp>
File "<stdin>", line 3, in _
NameError: name 'i' is not defined
>>> i = 6
>>> [f(0) for f in funcs]
[6, 6, 6]

doesn't make closures at all, but rather retains the global binding.

Totally different idiom again -- another strawman.

--Guido van Rossum (python.org/~guido)