lambda semantics in for loop

[Bengt Richter]

On 5 Jan 2003 04:36:39 -0800, henk at empanda.net (Henk Punt) wrote:

>Hi,
>
>When I have the following bit of python code:
>
>>>> l = []
>>>> for i in range(10):
>...     l.append(lambda x: x + i)
>
>I would expect:
>
>l[0](0) = 0
>l[1](0) = 1
>
>l[0](1) = 1
>l[1](1) = 2
>
>etc.
>
>instead
>
>l[0](0) = 9
>l[1](0) = 9
>
>l[0](1) = 10
>l[1](1) = 10
>
>It seems that the 'i' in the lambda binds to the last value of i in the
>for loop.
>Is this because 'i' is really a pointer and not the value of 'i' itself?.
>Please enlighten me!,
>
>How do I modify the example so that I would get my expected semantics.
>Should I copy 'i' to force the creation of a new object?, If so how would
>this work in the case where i is a string. I've tried to coerce python
>into making a deepcopy of a string so that id(s) != id(copy(s)) but I've
>not been able to do that also.
Martin has explained. I just would add that you have tools to peek at what
code is being created:

 >>> l = []
 >>> for i in range(10):
 ...     l.append(lambda x: x + i)
 ...
 >>> import dis
 >>> dis.dis(l[0])
           0 SET_LINENO               2
           3 LOAD_FAST                0 (x)
           6 LOAD_GLOBAL              1 (i)
           9 BINARY_ADD
          10 RETURN_VALUE
 >>> dis.dis(l[1])
           0 SET_LINENO               2
           3 LOAD_FAST                0 (x)
           6 LOAD_GLOBAL              1 (i)
           9 BINARY_ADD
          10 RETURN_VALUE
 >>> l[0](0)
 9
 >>> i=1000
 >>> l[0](0)
 1000

When you look at the code, the result is not so surprising.

Looking at Martin's default-value solution, you can see why it works:

 >>> l = []
 >>> for i in range(10):
 ...     l.append(lambda x, i = i: x + i)
 ...
 >>> l[0](0)
 0
 >>> l[1](0)
 1
 >>> dis.dis(l[1])
           0 SET_LINENO               2
           3 LOAD_FAST                0 (x)
           6 LOAD_FAST                1 (i)
           9 BINARY_ADD
          10 RETURN_VALUE
 
So what is the difference between LOAD_FAST (x) vs (i)?
The second is a default value, which you can determine
by looking at the list of defaults

 >>> l[1].func_defaults
 (1,)
 >>> l[0].func_defaults
 (0,)
 >>> l[2].func_defaults
 (2,)

and matching the list backards from the tail of the list of arguments,
of which there are two here:

 >>> l[2].func_code.co_argcount
 2

 >>> l[2].func_code.co_varnames
 ('x', 'i')

Which you read from the front of the list of varnames (here they're all args),
so you can now connect 'i' to its default value.

You could also have used a lambda to create another lambda with a closure containing
the value you wanted to capture:

 >>> l = []
 >>> for i in range(10):
 ...     l.append((lambda i: lambda x: x + i)(i))
 ...
 >>> l[0](0)
 0
 >>> l[1](0)
 1
 >>> l[2](0)
 2
 >>> dis.dis(l[2])
           0 SET_LINENO               2
           3 LOAD_FAST                0 (x)
           6 LOAD_DEREF               0 (i)
           9 BINARY_ADD
          10 RETURN_VALUE

No defaults this time:
 >>> l[2].func_defaults

Only the one arg:
 >>> l[2].func_code.co_varnames
 ('x',)
 >>> l[2].func_code.co_argcount
 1

But that LOAD_DEREF is getting a value someplace:

 >>> l[2].func_closure
 (<cell at 0x00855B60: int object at 0x00795988>,)

Knowing that small ints are shared, we can veriyfy what int
is encapsulated (I don't know off hand how to get the value out of
the cell)

 >>> hex(id(2))
 '0x795988'

AS expected.

Regards,
Bengt Richter