[Python-Dev] Explicit Lexical Scoping (pre-PEP?)

[Talin]

This is sort of a re-do of an earlier proposal which seems to have 
gotten lost in the shuffle of the larger debate.

I propose to create a new type of scoping rule, which I will call 
"explicit" lexical scoping, that will co-exist with the current 
"implicit" scoping rule that exists in Python today.


Definitions:

Implicit scoping is what we have now - a variable is defined within a 
scope implicitly by assignment. More specifically, when a name is 
assigned, the name is defined at the innermost function-level scope from 
where the assignment took place.

Explicit scoping is where the programmer explicitly specifies which 
scope the variable should be defined in. Unlike implicit scoping, 
assignments to a named variable do not automatically redefine that 
variable within the current scope.


Syntax:

Borrowing from Perl, the keyword 'my' is used to declare an explicitly 
scoped variable:

    def f1():
       my x = 1
       def f2():
          x = 2   # Does not create a new x

In the above example, the statement 'my x = 1' declares that the scope 
of the variable 'x' is the outer function f1. Any assignment to x will 
modify the existing x, rather than creating a new definition.

Note that the 'my' prefix can be combined with an assignment operation. 
It is anticipated that the 'my' prefix will be used quite frequently 
(and encouraged), so it makes sense to cut down on the number of 
statements by combining declaration and assignment.

Explicitly scoped variables can also be declared at the module level:

    my x = 1
    def f1():
       x = 2   # Modifies the global X

Declaring a module-level variable with an explicit scope eliminates the 
need for a 'global' statement for that variable.


Nested Scopes:

Each occurance of the keyword 'my' creates a new scope which hides any 
outer definitions of the name. So for example:

    my x = 1
    def f1():
       my x = 2  # This is a different 'x' than the global
       def f2():
          x = 3  # This is the 'x' defined within f1()


Interaction between explicit scoping and globals:

The 'global' statement, when used with explicitly scoped variables, 
means exactly the same as it does with implicitly scoped variables: It 
allows access to the outermost scope, overriding any intermediate 
definitions in surrounding scopes:

    x = 1
    def f1():
       my x = 2
       def f2():
          global x
          x = 3     # This is the module-level 'x'


Explicit scoping and code block structure:

Implicitly scoped variables are always defined at the nearest enclosing 
function scope, even if they are created within a code block.

It might be worth considering allowing explicitly scoped variables to be 
defined within other scopes. For example, we might choose to allow 
explicit scope declarations to be limited to the current suite:

    def f1():
       for x in range(0,10):
          my y = x*x          # A new definition of y for each iteration

Note that this is a speculation only, and not a core part of the 
proposal (so please don't reject the proposal on this one point.)


Formal definition:

When a value is assigned to a local variable name, the rules for 
determining which scope the variable will be defined in are as follows:

    1) Starting with the current (innermost) scope, examine all of the 
currently active scopes:
       1a) If the current scope contains a 'global' statement for the 
given name, then set the result scope to the outermost (module-level) scope.
       1b) If the current scope contains a 'my' statement for the given 
name, then set the result scope to the scope in which the 'my' statement 
occurred.
    2) Otherwise, continue until we run out of scopes. If neither a 
'global' or 'my' declaration was discovered, then use the innermost 
scope as the result scope.


How is this different from 'outer'?

The explicit scope proposal requires that the scope be specified at the 
place where the variable is *defined* as opposed to where it is *used*. 
This definition is inherited by all inner scopes.

This allows a finer degree of control, for less typing, than the 'outer' 
proposal. With explicit scoping, there is no confusion as to which scope 
is being considered; And explicit scoping allows a single declaration of 
a variable to be shared by many different inner scopes, which would 
otherwise require a separate 'outer' statement for each one.


Explicit scoping and static analysis:

It should be easier to do static analysis of code with explicit scoping, 
since you always know what scope a variable is defined in (as opposed to 
implicit scoping, where a variable may switch from global to local as a 
result of an assignment.)

Note that this implies that the creation of the scope does not occur at 
the time of the assignment, but rather at the time the function is 
entered. Thus:

    x = 1
    def f1():
       print x   # Error, unassigned value
       my x = 2

In the above example, even though the 'my' statement occurs after the 
print, the scope created by the 'my' statement is in effect for the 
entire function, although the actual *assignment* takes place after the 
print. The reason for this is that the scope creation is actually done 
by the compiler.

-- Talin