Confused about closures and scoping rules

[Diez B. Roggisch]

Fernando Perez schrieb:
> Hi all,
> 
> consider the following small example:
> 
> """
> Small test to try to understand a strange subtlety with closures
> """
> 
> def outer(nmax):
> 
>     aa = []
>     for n in range(nmax):
> 
>         def a(y):
>             return (y,n)
>         print 'Closure and cell id:',id(a.func_closure),\
>               id(a.func_closure[0])
>         aa.append(a)
> 
>     return aa
> 
> print 'Closure creation.'
> nmax = 3
> aa = outer(nmax)
> 
> print
> print 'Closure use.'
> for n in range(nmax):
>     print '%s:%s' % (n,aa[n]('hello'))
> 
> ################## EOF #################
> 
> 
> If I run this, I get:
> 
> planck[test]> python debug_closures.py
> Closure creation.
> Closure and cell id: 1075998828 1075618940
> Closure and cell id: 1075999052 1075618940
> Closure and cell id: 1075999084 1075618940
> 
> Closure use.
> 0:('hello', 2)
> 1:('hello', 2)
> 2:('hello', 2)
> 
> 
> My confusion arises from the printout after 'closure use'. I was expecting that
> each new function 'a' created inside the loop in 'outer' would capture the
> value of n, therefore my expectation was to see a printout like:
> 
> 0:('hello', 0)
> 1:('hello', 1)... etc.
> 
> However, what happens is a bit different.  As can be seen from the printouts
> of 'Closure and cell id', in each pass of the loop a new closure is created,
> but it reuses the *same* cell object every time.  For this reason, all the
> closures end up sharing the scope with the values determined by the *last*
> iteration of the loop.
> 
> This struck me as counterintuitive, but I couldn't find anything in the
> official docs indicating what the expected behavior should be. Any
> feedback/enlightenment would be welcome.  This problem appeared deep inside a
> complicated code and it took me almost two days to track down what was going
> on...

It's a FAQ. The reason is that the created closures don't capture the 
_value_, but the _name_. Plus of course the locals()-dictionary outside 
the function a to perform the lookup of that name. Which has the value 
bound to it in the last iteration.

Common cure for this is to create an a-local name that shadows the outer 
variable and is simultaneously bound to the desired value:

def outer(nmax):

     aa = []
     for n in range(nmax):
         foo = 'bar'
         def a(y,n=n):
             bar = foo
             return (y,n)
         print 'Closure and cell id:',id(a.func_closure),\
               id(a.func_closure[0])
         aa.append(a)

     return aa

print 'Closure creation.'
nmax = 3
aa = outer(nmax)

print
print 'Closure use.'
for n in range(nmax):
     print '%s:%s' % (n,aa[n]('hello'))


Notice the foo/bar - that was necessary to actually create a closure at 
all (to keep your printing working), as python statically checks if 
there needs one to be.



Diez