Something is rotten in Denmark...

[Steven D'Aprano]

On Wed, 01 Jun 2011 19:40:30 -0500, harrismh777 wrote:

> The part that I don't see much about in the docs (some books, that is)
> is that the lambda lookups occur late (the lambda is evaluated at the
> time it is called). The Python docs on-line *do say* this (I found too
> late) but its one quick phrase that can be missed. So, the i in
> range(10) is sitting there at '9' by the time *any* of the ten lambdas
> get called. This is not intuitive, nor good. IMHO

I agree it's not intuitive. But where does it say that programming 
language semantics must always be intuitive? Whose intuition? Mine? 
Yours? Linus Torvalds'? Donald Knuth's? My auntie Rose's?


> Please allow me to whine a little bit, ... but the *whole point* of
> iterating is to be able to implicitly grab each iterated value as it
> flies by (by the lambda or anything else!) and there is not much point
> to having a 'late-binding' on an iterable particularly range(n).

What do you expect this code to do?

a = 42
funcs = [(lambda x: x+a) for i in range(10)]
funcs[0](1)

a = 23
funcs[0](1)


Do you agree that `a` should be late bound in this situation?

If so, why do you think that `i` should be early bound here?

funcs = [(lambda x: x+i) for i in range(10)]


Oh, the fact that it works at all in Python 2.5 is a side-effect of i 
leaking from the list comprehension:

>>> funcs = [(lambda x: x+i) for i in range(10)]
>>> del i
>>> funcs[0](1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in <lambda>
NameError: global name 'i' is not defined


We can see that the closure isn't created:

>>> funcs[0].func_closure is None
True


However, with a generator expression, i does not leak, a closure is 
created, but it is still late bound:

>>> funcs = list((lambda x: x+i) for i in range(10))
>>> funcs[0].func_closure
(<cell at 0xb7ed44dc: int object at 0x8121ed0>,)
>>> del i
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'i' is not defined
>>> funcs[0](1)
10
>>> funcs[1](1)
10


> Yes, I can explicitly grab each 'i' as it flies by with a little clever
> coding of the default value for the  lambda n, i=i: i + n  but that
> 'trick' is not intuitive, nor is it clear reading. It 'works' is just
> about all one can say for it (not very elegant).

It might be more clear reading if you do it this way:

funcs = [(lambda x, i=j: x+i) for j in range(10)]

Now the reader is no longer distracted by the "i=i" ugliness.


> I'm not sure what the answer is, but I think all of us need to think
> through it some more. Placing lambdas in a list comprehension is just
> delicious, except for the explicit kludges we have to code to get it to
> work. I'm wondering if whether it would make some sense to put some
> 'binding smarts' into the interpreter to allow for 'interpreter
> intuition' (say AI ) that would presume to understand when early vs late
> binding makes sense and apply early binding in those cases where the
> context is not ambiguous and when it is clear that an iterable is being
> passed to the constant lambda function??

The problem with Do What I Mean is that it so rarely Does What You Mean. 
At best it Does What Some Other Guy Imagined I'd Probably Mean In This 
Situation. Let's not go there.


-- 
Steven