strange behavor....

[Steven D'Aprano]

On Sat, 13 Nov 2010 22:22:00 +0000, Mark Wooding wrote:

> Challenge: explain the following code using only those concepts.

("those concepts" being name and value/object)

>         def foo():
>           l = []
>           for i in xrange(10):
>             (lambda j: l.append((lambda: i, lambda: j)))(i)
>           print [(f(), g()) for f, g in l]

>>> foo()
[(9, 0), (9, 1), (9, 2), (9, 3), (9, 4), (9, 5), (9, 6), (9, 7), (9, 8), 
(9, 9)]


Here's a slightly less condensed version that demonstrates the same 
behaviour, but may be slightly easier to understand:

def spam():
    a = []
    b = []
    for i in xrange(5):
        a.append(lambda: i)
        b.append(lambda i=i: i)
    print [f() for f in a]
    print [f() for f in b]

Anyway, here goes...

In your function foo, the name "i" is bound to the objects 0, 1, 2, ... 9 
sequentially, each time around the for-loop. Inside the loop, a function 
object is created and then called:

(lambda j: l.append((lambda: i, lambda: j)))(i)

(Aside: this depends on scoping rules that were introduced quite late in 
Python's history -- prior to version 2.2, this wouldn't work at all.)

The "outer" lambda:

    lambda j: l.append(...)

has a name in the function namespace (a "local variable") called "j". On 
entry to this function, this j is bound to the same object which is bound 
to i *at the time that the function is called*. That is, each of the 
sequence of "outer" lambdas get a distinct j = 0, 1, 2, ... 

What does these outer lambdas do? They create two more function objects, 
which I will label "A" and "B":

A:: lambda: i
B:: lambda: j

and places those function objects in a tuple and appends the tuple to the 
list. What do these "inner" functions do? They have no local variable of 
their own -- they refer only the i and j of their enclosing scope, namely 
the "outer" lambda that created them.

Note that there isn't actually a single A and a single B, there is a 
whole series of them... we might label them A0, A1, A2, ... etc.

None of the function A0, A1, A2, ... have any local variable i. When you 
call them, Python looks in the enclosing scopes for a variable i. It 
doesn't find one in the "outer" lambda, so it next searches the namespace 
of foo, where it finds the name i bound to the object 9. Hence the inner 
lambdas Ai always return 9, since 9 is the value of i when the function 
is called.

Functions B0, B1, B2, ... similarly have no local variable j. When you 
call any of the Bs, Python looks in the enclosing scopes for a variable 
j. However, in this case it *does* find one, in the "outer" lambda, where 
the name j has been bound to some object which was determined at the time 
"outer" was called, namely the value of i *at the time*, and hence 0, 1, 
2, ... depending on which specific function you're looking at. Hence the 
inner lambda Bi returns the value of i *at the time it was defined*.

 
> I explain this as follows.
> 
>   * Python's `for' loop works by assignment.  The name `i' remains bound
>     to the same storage location throughout; 

This is not necessarily true. It's true for CPython, where function 
locals are implemented as fixed slots in the function object; since the 
slot doesn't move relative to the function, and the function doesn't move 
relative to the heap, the name i is in a fixed storage location for the 
life of foo. But that's not necessarily the case for all implementations 
-- locals could be stored in a data structure that moves data around 
(say, a red-black tree), or objects could be free to move in the heap, or 
both.



-- 
Steven