[Tutor] Python scope and variable binding

[eryksun]

On Wed, Nov 27, 2013 at 10:04 AM, Arnaud Legout <arnaud.legout at inria.fr> wrote:
>
> example 4:
>
> x = "x in module"
> class A():
>     print "A: " + x
>     x = "x in A"
>     print "A: " + x
>     print locals()
>     del x
>     print locals()
>     print "A: " + x
>>>>
> A: x in module
> A: x in A
> {'x': 'x in A', '__module__': '__main__'}
> {'__module__': '__main__'}
> A: x in module
>
> But we see here that this statement in PEP227 "If a name binding
> operation occurs anywhere within a code block, all uses of the name
> within the block are treated as references to the current block." is
> wrong when the code block is a class. Moreover, for classes, it seems
> that local name binding is not made at compile time, but during
> execution using the class namespace in __dict__. In that respect,
> PEP227 and the execution model in the Python doc is misleading and for
> some parts wrong.

In a module or class, even if a name is local, loading still falls
back on globals and builtins. So a local name can temporarily shadow a
name from globals or builtins, and once deleted the original is
available again.

That can't be done in function code that uses fast locals. Loading a
fast local doesn't fall back to globals and builtins. For example,
this initial assignment will fail: int = int. You might instead use
int_ = int, or a default argument.

PEP 227 is concerned with nested scopes that support lexical closures.
The implementation associates a name with a cell object that can be
shared across scopes.

CPython bytecode operations:

LOAD_NAME - locals, globals, builtins
STORE_NAME - locals
LOAD_FAST, STORE_FAST - fast locals array
LOAD_DEREF, STORE_DEREF - local cellvar, nonlocal freevar
LOAD_GLOBAL - globals, builtins
STORE_GLOBAL - globals

In the following example, dtype is a free variable in g and a cell
variable in f.

    def f(dtype):
        def g(x):
            return dtype(x)
        return g

    g_int = f(int)

    >>> f.__code__.co_cellvars
    ('dtype',)
    >>> g_int.__code__.co_freevars
    ('dtype',)

    >>> type(g_int.__closure__[0])
    <type 'cell'>
    >>> g_int.__closure__[0].cell_contents
    <type 'int'>

    >>> dis.dis(g_int)
      3           0 LOAD_DEREF               0 (dtype)
                  3 LOAD_FAST                0 (x)
                  6 CALL_FUNCTION            1
                  9 RETURN_VALUE

Versions prior to 2.6 use the function attributes func_code and
func_closure instead of __code__ and __closure__. The old attribute
names are still available in 2.6/2.7, but not in 3.x.

3.x adds a nonlocal statement to declare free variables. Can you infer
what this enables?


> example 5:
>
> x = 'x in module'
> def  f2():
>     x = 'x in f2'
>     def myfunc():
>         x = 'x in myfunc'
>         class MyClass(object):
>             x = x
>             print x
>         return MyClass
>     myfunc()
> f2()
>>>>
> x in module
>
> my understanding of this code is the following. The instruction x = x
> first look up the object the right hand x of the expression is referring
> to. In that case, the object is looked up locally in the class, then
> following the rule LGB it is looked up in the global scope, which is
> the string 'x in module'. Then a local attribute x to MyClass is
> created in the class dictionary and pointed to the string object.

Regarding the last sentence, basically the locals dict from the frame
that executed the class body is passed to the metaclass, which stores
a copy to the new type. If you've never created a type manually by
calling a metaclass, here's an example.

    # type(name, bases, dict) -> a new type

    C = type('C', (object,), {'x': 'x in C'})

    >>> C.__name__
    'C'
    >>> C.__bases__
    (<type 'object'>,)
    >>> C.x
    'x in C'


> example 6:
> Now here is an example I cannot explain.
> It is very close to example 5, I am just changing the local MyClass
> attribute from x to y.
>
> x = 'x in module'
> def  f2():
>     x = 'x in f2'
>     def myfunc():
>         x = 'x in myfunc'
>         class MyClass(object):
>             y = x
>             print y
>         return MyClass
>     myfunc()
> f2()
>>>>
> x in myfunc
>
> Why in that case the x reference in MyClass is looked up in the
> innermost function?

You aren't assigning x in the class body, so it's a free variable
there, and a cell variable in myfunc. You could just use "print x" to
get the same result.

Let's try something similar in 3.x but with an explicit nonlocal
declaration (allowing reassignment of a free variable):

    def f():
        x = 'x in f'
        print('f,1:', x)
        class C(object):
            nonlocal x
            print('C,1:', x)
            x = 'x in C'
            print('C,2:', x)
        print('f,2:', x) # reassigned in C

    >>> f()
    f,1: x in f
    C,1: x in f
    C,2: x in C
    f,2: x in C

Now again without the nonlocal declaration:

    x = 'x in module'

    def f():
        x = 'x in f'
        print('f,1:', x)
        class C(object):
            print('C,1:', x)
            x = 'x in C'
            print('C,2:', x)
        print('f,2:', x)

    >>> f()
    f,1: x in f
    C,1: x in module
    C,2: x in C
    f,2: x in f

Without a global or nonlocal declaration, assignment implicitly makes
x a local in the class body. The C,1 LOAD_NAME finds x in globals. The
C,2 LOAD_NAME finds x in locals.