Odd closure issue for generators

Carl Banks pavlovevidence at gmail.com
Thu Jun 4 19:18:32 EDT 2009


On Jun 4, 2:40 pm, Brian Quinlan <br... at sweetapp.com> wrote:
> This is from Python built from the py3k branch:
>  >>> c = (lambda : i for i in range(11, 16))
>  >>> for q in c:
> ...     print(q())
> ...
> 11
> 12
> 13
> 14
> 15
>  >>> # This is expected
>  >>> c = (lambda : i for i in range(11, 16))
>  >>> d = list(c)
>  >>> for q in d:
> ...     print(q())
> ...
> 15
> 15
> 15
> 15
> 15
>  >>> # I was very surprised
>
> Looking at the implementation, I see why this happens:
>  >>> c = (lambda : i for i in range(11, 16))
>  >>> for q in c:
> ...     print(id(q.__closure__[0]))
> ...
> 3847792
> 3847792
> 3847792
> 3847792
> 3847792
>  >>> # The same closure is used by every lambda
>
> But it seems very odd to me and it can lead to some problems that are a
> real pain in the ass to debug.

It's really the only sane way to handle it, odd though it may seem in
this narrow case.  In Python nested functions have to be able to
reference the current value of a variable because of use cases like
this:

def func():
    def printx():
        print x
    x = 1
    printx()
    x = 2
    printx()

Referencing a nonlocal variable always uses the current (or last)
value of that variable, not the value it had when the nested function
was defined.


The way to handle the issue you are seeing is to create a new scope
with a variable the remains at the value you want to close upon:

create_const_function(value):
    def func():
        return value
c = (create_const_function(i) for i in range(11, 16))

Or you can do it the slacker way and use a default argument:

c = (lambda i=i: i for i in range(11, 16))


Carl Banks



More information about the Python-list mailing list