unintuitive for-loop behavior

[Steve D'Aprano]

On Sun, 2 Oct 2016 03:57 am, Rustom Mody wrote:

> Hoo boy1
> Thats some tour de force and makes my head spin

I certainly agree with the second part of your sentence.


> Point can be made more simply with map
> ie if we *define*
> [exp for cv in l]
> as
> map(lambda cv: exp, l)
> 
> the problem vanishes

> 
> Demo:
> 
> First a helper function for demoing:
> 
> def pam(fl,x):
>     return map(lambda f: f(x), fl)
> # pam is the complement to map; map runs one fnc on a list of args
> # pam runs a list of funcs on one arg
> 
> Trying to make a list of functions that add one, two and three to their
> arguments
> 
> fl = [lambda x: x + cv for cv in [1,2,3]]
> 
> Broken because of python's wrong LC semantics:
>>>> pam(fl, 3)
> [6, 6, 6]

Its not *broken*, its doing *exactly what you told it to do*. You said,
define a function that takes a single argument x, and return x + cv. Then
you delayed evaluating it until cv = 3, and passed the argument 3, so of
course it returns 6. That's exactly what you told it to calculate.

You seem to have the concept that lambda should be magical, and just
miraculously know how far back in time to look for the value of cv. And
then when it doesn't, you're angry that Python is "broken". But why should
it be magical? cv is just an ordinary variable, and like all variables,
looking it up returns the value it has at the time you do the look-up, not
some time in the past. Let's unroll the loop:

fl = []
cv = 1
def f(x): return x + cv
fl.append(f)
cv = 2
def f(x): return x + cv
fl.append(f)
cv = 3
def f(x): return x + cv
fl.append(f)

pam(fl, 3)

Are you still surprised that it returns [6, 6, 6]?




> Transform the LC into a map with the rule above:
> fl_good = map((lambda cv :lambda x: x+cv), [1,2,3])


This is equivalent to something completely different, using a closure over
cv, so of course it works:

def factory(cv):
    def inner(x):
        return x + cv
    return inner

fl_good = []
fl_good.append(factory(1))
fl_good.append(factory(2))
fl_good.append(factory(3))


Each time you call factory(), you get a new scope, with its own independent
variable cv. The inner function captures that environment (a closure),
which includes that local variable cv. Each invocation of factory leads to
an inner function that sees a different local variable which is independent
of the others but happens to have the same name. Instead of three functions
all looking up a single cv variable, you have three functions looking up
three different cv variables.

This is essentially why closures exist.


> Which is not very far from the standard workaround for this gotcha:
>>>> fl_workaround = [lambda x, cv=cv: x+cv for cv in [1,2,3]]
>>>> pam(fl_workaround, 3)
> [4, 5, 6]
>>>> 
> 
> Maybe we could say the workaround is the map definition uncurried
> And then re-comprehension-ified

If your students think in terms of map, then fine, but I think it would
confuse more people than it would help. Your mileage may vary.

There are certainly a number of ways to get the desired behaviour. If you
prefer to work with map, go right ahead.




-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.