[Python-Dev] statically nested scopes

[Jeremy Hylton]

Title: Statically Nested Scopes
Author: Jeremy Hylton <jeremy@digicool.com>
Status: Draft
Type: Standards Track
Created: 01-Nov-2000

Abstract

    This PEP proposes the additional of statically nested scoping
    (lexical scoping) for Python 2.1.  The current language definition
    defines exactly three namespaces that are used to resolve names --
    the local, global, and built-in namespaces.  The addition of
    nested scopes would allow resolution of unbound local names in
    enclosing functions' namespaces.

    One consequence of this change that will be most visible to Python
    programs is that lambda statements could reference variables in
    the namespaces where the lambda is defined.  Currently, a lambda
    statement uses default arguments to explicitly creating bindings
    in the lambda's namespace.

Notes

    This section describes several issues that will be fleshed out and
    addressed in the final draft of the PEP.  Until that draft is
    ready, please direct comments to the author.

    This change has been proposed many times in the past.  It has
    always been stymied by the possibility of creating cycles that
    could not be collected by Python's reference counting garbage
    collector.  The additional of the cycle collector in Python 2.0
    eliminates this concern.

    Guido once explained that his original reservation about nested
    scopes was a reaction to their overuse in Pascal.  In large Pascal
    programs he was familiar with, block structure was overused as an
    organizing principle for the program, leading to hard-to-read
    code. 

    Greg Ewing developed a proposal "Python Nested Lexical Scoping
    Enhancement" in Aug. 1999.  It is available from
    http://www.cosc.canterbury.ac.nz/~greg/python/lexscope.html

    Michael Hudson's bytecodehacks projects at
    http://sourceforge.net/projects/bytecodehacks/
    provides facilities to support nested scopes using the closure
    module. 

    Examples:

    def make_adder(n):
        def adder(x):
            return x + n
        return adder
    add2 = make_adder(2)
    add2(5) == 7

    
    from Tkinter import *
    root = Tk()
    Button(root, text="Click here",
           command = lambda : root.test.configure(text="..."))

    
    One controversial issue is whether it should be possible to modify
    the value of variables defined in an enclosing scope.  

    One part of the issue is how to specify that an assignment in the
    local scope should reference to the definition of the variable in
    an enclosing scope.  Assignment to a variable in the current scope
    creates a local variable in the scope.  If the assignment is
    supposed to refer to a global variable, the global statement must
    be used to prevent a local name from being created.  Presumably,
    another keyword would be required to specify "nearest enclosing
    scope."

    Guido is opposed to allow modifications (need to clarify exactly
    why).  If you are modifying variables bound in enclosing scopes,
    you should be using a class, he says.

    The problem occurs only when a program attempts to rebind the name
    in the enclosing scope.  A mutable object, e.g. a list or
    dictionary, can be modified by a reference in a nested scope; this
    is an obvious consequence of Python's reference semantics.  The
    ability to change mutable objects leads to an inelegant
    workaround: If a program needs to rebind an immutable object,
    e.g. a number or tuple, store the object in a list and have all
    references to the object use this list:

    def bank_account(initial_balance):
        balance = [initial_balance]
        def deposit(amount):
            balance[0] = balance[0] + amount
        def withdraw(amount):
            balance[0] = balance[0] - amount
        return deposit, withdraw

    I would prefer for the language to support this style of
    programming directly rather than encouraging programs to use this
    somewhat obfuscated style.  Of course, an instance would probably
    be clearer in this case.
 
    One implementation issue is how to represent the environment that
    stores variables that are referenced by nested scopes.  One
    possibility is to add a pointer to each frame's statically
    enclosing frame and walk the chain of links each time a non-local
    variable is accessed.  This implementation has some problems,
    because access to nonlocal variables is slow and causes garbage to
    accumulate unncessarily.  Another possibility is to construct an
    environment for each function that provides access to only the
    non-local variables.  This environment would be explicitly passed
    to nested functions.