[Tutor] question about metaclasses

[Steven D'Aprano]

On Wed, Jan 10, 2018 at 07:29:58PM +0100, Peter Otten wrote:

[...]
>             elif not isinstance(obj, property):
>                 attrs[attr] = property(lambda self, obj=obj: obj)

> PS: If you don't remember why the obj=obj is necessary:
> Python uses late binding; without that trick all lambda functions would 
> return the value bound to the obj name when the for loop has completed.

This is true, but I think your terminology is misleading. For default 
values to function parameters, Python uses *early* binding, not late 
binding: the default value is computed once at the time the function is 
created, not each time it is needed.

So in this case, each of those property objects use a function that sets 
the default value of obj to the current value of obj at the time that 
the property is created.

Without the obj=obj parameter, Python creates a *closure*. A closure is 
a computer-science term for something like a snap shot of the 
environment where the function was created.

If we had written this instead:

    attrs[attr] = property(lambda self: obj)

the name "obj" doesn't refer to a local variable of the lambda function. 
Nor does it refer to a global variable, or a builtin function. It refers 
to a "non-local variable": it belongs to the function that surrounds the 
lambda function, not the lambda itself.

And so Python would create a *closure* for the lambda function so that 
when it eventually gets called, it knows where to find the value of obj.

And *that* process, of looking up the value of obj from a closure, uses 
late binding: all the lambda functions will refer to the same 
environment, which means they will also see the same value for obj.

Namely the last value obj received when the outer function (the one 
that the closure refers back to) completed.

Here's another example to show the difference. Rather than use lambda, 
I'm going to use regular "def" to prove that this has nothing to do with 
lambda itself, the rules apply every time you create a function.

Start with the closure version:

# --- cut here %< ---

def factory_closure():
    # Create many new functions, each of which refer to i in its
    # enclosing scope.
    functions = []
    for i in range(5):
        def f():
            return i  # << this i is a NONLOCAL variable
        functions.append(f)
    return functions

functions = factory_closure()

# All the closures refer to the same thing.
for f in functions:
    print(f.__closure__)

# And the functions all see the same value for i
print([f() for f in functions])

# --- cut here %< ---


And here is a version which avoids the closure issue by using the 
function parameter default value trick:


# --- cut here %< ---

def factory_no_closure():
    # Create many new functions, each of which refer to i using
    # a parameter default value.
    functions = []
    for i in range(5):
        def f(i=i):
            return i  # << this i is a LOCAL variable
        functions.append(f)
    return functions

functions = factory_no_closure()

# None of the functions need a closure.
for g in functions:
    print(g.__closure__)

# And the functions all see different values for i
print([g() for g in functions])

# --- cut here %< ---


In practice, this is generally only an issue when single invocation of a 
factory function creates two or more functions at once, and that 
generally means inside a loop:


def factory():
    for i in something:
        create function referring to i


If your factory only returns one function at a time, like this:


def factory(i):
    create function referring to i

for n in something:
    factory(n)


then each function still uses a closure, but they are *different* 
closures because each one is created on a different invocation of the 
factory. That's another way to avoid this "early/late binding" problem.


-- 
Steve