[Python-checkins] r52701 - peps/trunk/pep-3104.txt
ka-ping.yee
python-checkins at python.org
Thu Nov 9 20:53:17 CET 2006
Author: ka-ping.yee
Date: Thu Nov 9 20:53:16 2006
New Revision: 52701
Added:
peps/trunk/pep-3104.txt
Log:
Add PEP 3104 (Access to Names in Outer Scopes).
Added: peps/trunk/pep-3104.txt
==============================================================================
--- (empty file)
+++ peps/trunk/pep-3104.txt Thu Nov 9 20:53:16 2006
@@ -0,0 +1,537 @@
+PEP: 3104
+Title: Access to Names in Outer Scopes
+Version: $Revision$
+Last-Modified: $Date: 2006-11-09 11:52:55 -0800 (Thu, 9 Nov 2006) $
+Author: Ka-Ping Yee <ping at zesty.ca>
+Status: Draft
+Type: Standards Track
+Python-Version: 3.0
+Content-Type: text/x-rst
+Created: 12-Oct-2006
+
+
+Abstract
+========
+
+In most languages that support nested scopes, code can refer to or
+rebind (assign to) any name in the nearest enclosing scope.
+Currently, Python code can refer to a name in any enclosing scope,
+but it can only rebind names in two scopes: the local scope (by
+simple assignment) or the module-global scope (using a ``global``
+declaration).
+
+This limitation has been raised many times on the Python-Dev mailing
+list and elsewhere, and has led to extended discussion and many
+proposals for ways to remove this limitation. This PEP summarizes
+the various alternatives that have been suggested, together with
+advantages and disadvantages that have been mentioned for each.
+
+
+Rationale
+=========
+
+Before version 2.1, Python's treatment of scopes resembled that of
+standard C: within a file there were only two levels of scope, global
+and local. In C, this is a natural consequence of the fact that
+function definitions cannot be nested. But in Python, though
+functions are usually defined at the top level, a function definition
+can be executed anywhere. This gave Python the syntactic appearance
+of nested scoping without the semantics, and yielded inconsistencies
+that were surprising to some programmers -- for example, a recursive
+function that worked at the top level would cease to work when moved
+inside another function, because the recursive function's own name
+would no longer be visible in its body's scope. This violates the
+intuition that a function should behave consistently when placed in
+different contexts. Here's an example::
+
+ def enclosing_function():
+ def factorial(n):
+ if n < 2:
+ return 1
+ return n * factorial(n - 1) # fails with NameError
+ print factorial(5)
+
+Python 2.1 moved closer to static nested scoping by making visible
+the names bound in all enclosing scopes (see PEP 227). This change
+makes the above code example work as expected. However, because any
+assignment to a name implicitly declares that name to be local, it is
+impossible to rebind a name in an outer scope (except when a
+``global`` declaration forces the name to be global). Thus, the
+following code, intended to display a number that can be incremented
+and decremented by clicking buttons, doesn't work as someone familiar
+with lexical scoping might expect::
+
+ def make_scoreboard(frame, score=0):
+ label = Label(frame)
+ label.pack()
+ for i in [-10, -1, 1, 10]:
+ def increment(step=i):
+ score = score + step # fails with UnboundLocalError
+ label['text'] = score
+ button = Button(frame, text='%+d' % i, command=increment)
+ button.pack()
+ return label
+
+Python syntax doesn't provide a way to indicate that the name
+``score`` mentioned in ``increment`` refers to the variable ``score``
+bound in ``make_scoreboard``, not a local variable in ``increment``.
+Users and developers of Python have expressed an interest in removing
+this limitation so that Python can have the full flexibility of the
+Algol-style scoping model that is now standard in many programming
+languages, including JavaScript, Perl, Ruby, Scheme, Smalltalk,
+C with GNU extensions, and C# 2.0.
+
+It has been argued that that such a feature isn't necessary, because
+a rebindable outer variable can be simulated by wrapping it in a
+mutable object::
+
+ class Namespace:
+ pass
+
+ def make_scoreboard(frame, score=0):
+ ns = Namespace()
+ ns.score = 0
+ label = Label(frame)
+ label.pack()
+ for i in [-10, -1, 1, 10]:
+ def increment(step=i):
+ ns.score = ns.score + step
+ label['text'] = ns.score
+ button = Button(frame, text='%+d' % i, command=increment)
+ button.pack()
+ return label
+
+However, this workaround only highlights the shortcomings of existing
+scopes: the purpose of a function is to encapsulate code in its own
+namespace, so it seems unfortunate that the programmer should have to
+create additional namespaces to make up for missing functionality in
+the existing local scopes, and then have to decide whether each name
+should reside in the real scope or the simulated scope.
+
+Another common objection is that the desired functionality can be
+written as a class instead, albeit somewhat more verbosely. One
+rebuttal to this objection is that the existence of a different
+implementation style is not a reason to leave a supported programming
+construct (nested scopes) functionally incomplete. Python is
+sometimes called a "multi-paradigm language" because it derives so
+much strength, practical flexibility, and pedagogical power from its
+support and graceful integration of multiple programming paradigms.
+
+A proposal for scoping syntax appeared on Python-Dev as far back as
+1994 [1]_, long before PEP 227's support for nested scopes was
+adopted. At the time, Guido's response was:
+
+ This is dangerously close to introducing CSNS [classic static
+ nested scopes]. *If* you were to do so, your proposed semantics
+ of scoped seem allright. I still think there is not enough need
+ for CSNS to warrant this kind of construct ...
+
+After PEP 227, the "outer name rebinding discussion" has reappeared
+on Python-Dev enough times that it has become a familiar event,
+having recurred in its present form since at least 2003 [2]_.
+Although none of the language changes proposed in these discussions
+have yet been adopted, Guido has acknowledged that a language change
+is worth considering [12]_.
+
+
+Other Languages
+===============
+
+To provide some background, this section describes how some other
+languages handle nested scopes and rebinding.
+
+JavaScript, Perl, Scheme, Smalltalk, GNU C, C# 2.0
+--------------------------------------------------
+
+These languages use variable declarations to indicate scope. In
+JavaScript, a lexically scoped variable is declared with the ``var``
+keyword; undeclared variable names are assumed to be global. In
+Perl, a lexically scoped variable is declared with the ``my``
+keyword; undeclared variable names are assumed to be global. In
+Scheme, all variables must be declared (with ``define`` or ``let``,
+or as formal parameters). In Smalltalk, any block can begin by
+declaring a list of local variable names between vertical bars.
+C and C# require type declarations for all variables. For all these
+cases, the variable belongs to the scope containing the declaration.
+
+Ruby (as of 1.8)
+----------------
+
+Ruby is an instructive example because it appears to be the only
+other currently popular language that, like Python, tries to support
+statically nested scopes without requiring variable declarations, and
+thus has to come up with an unusual solution. Functions in Ruby can
+contain other function definitions, and they can also contain code
+blocks enclosed in curly braces. Blocks have access to outer
+variables, but nested functions do not. Within a block, an
+assignment to a name implies a declaration of a local variable only
+if it would not shadow a name already bound in an outer scope;
+otherwise assignment is interpreted as rebinding of the outer name.
+Ruby's scoping syntax and rules have also been debated at great
+length, and changes seem likely in Ruby 2.0 [25]_.
+
+
+Overview of Proposals
+=====================
+
+There have been many different proposals on Python-Dev for ways to
+rebind names in outer scopes. They all fall into two categories:
+new syntax in the scope where the name is bound, or new syntax in
+the scope where the name is used.
+
+New Syntax in the Binding (Outer) Scope
+---------------------------------------
+
+Scope Override Declaration
+''''''''''''''''''''''''''
+
+The proposals in this category all suggest a new kind of declaration
+statement similar to JavaScript's ``var``. A few possible keywords
+have been proposed for this purpose:
+
+ - ``scope x`` [4]_
+ - ``var x`` [4]_ [9]_
+ - ``my x`` [13]_
+
+In all these proposals, a declaration such as ``var x`` in a
+particular scope S would cause all references to ``x`` in scopes
+nested within S to refer to the ``x`` bound in S.
+
+The primary objection to this category of proposals is that the
+meaning of a function definition would become context-sensitive.
+Moving a function definition inside some other block could cause any
+of the local name references in the function to become nonlocal, due
+to declarations in the enclosing block. For blocks in Ruby 1.8,
+this is actually the case; in the following example, the two setters
+have different effects even though they look identical::
+
+ setter1 = proc { | x | y = x } # y is local here
+ y = 13
+ setter2 = proc { | x | y = x } # y is nonlocal here
+ setter1.call(99)
+ puts y # prints 13
+ setter2.call(77)
+ puts y # prints 77
+
+Note that although this proposal resembles declarations in JavaScript
+and Perl, the effect on the language is different because in those
+languages undeclared variables are global by default, whereas in
+Python undeclared variables are local by default. Thus, moving
+a function inside some other block in JavaScript or Perl can only
+reduce the scope of a previously global name reference, whereas in
+Python with this proposal, it could expand the scope of a previously
+local name reference.
+
+Required Variable Declaration
+'''''''''''''''''''''''''''''
+
+A more radical proposal [21]_ suggests removing Python's scope-guessing
+convention altogether and requiring that all names be declared in the
+scope where they are to be bound, much like Scheme. With this
+proposal, ``var x = 3`` would both declare ``x`` to belong to the
+local scope and bind it, where as ``x = 3`` would rebind the existing
+visible ``x``. In a context without an enclosing scope containing a
+``var x`` declaration, the statement ``x = 3`` would be statically
+determined to be illegal.
+
+This proposal yields a simple and consistent model, but it would be
+incompatible with all existing Python code.
+
+New Syntax in the Referring (Inner) Scope
+-----------------------------------------
+
+There are three kinds of proposals in this category.
+
+Outer Reference Expression
+''''''''''''''''''''''''''
+
+This type of proposal suggests a new way of referring to a variable
+in an outer scope when using the variable in an expression. One
+syntax that has been suggested for this is ``.x`` [7]_, which would
+refer to ``x`` without creating a local binding for it. A concern
+with this proposal is that in many contexts ``x`` and ``.x`` could
+be used interchangeably, which would confuse the reader. A closely
+related idea is to use multiple dots to specify the number of scope
+levels to ascend [8]_, but most consider this too error-prone [17]_.
+
+Rebinding Operator
+''''''''''''''''''
+
+This proposal suggests a new assignment-like operator that rebinds
+a name without declaring the name to be local [2]_. Whereas the
+statement ``x = 3`` both declares ``x`` a local variable and binds
+it to 3, the statement ``x := 3`` would change the existing binding
+of ``x`` without declaring it local.
+
+This is a simple solution, but according to PEP 3099 it has been
+rejected (perhaps because it would be too easy to miss or to confuse
+with ``=``).
+
+Scope Override Declaration
+''''''''''''''''''''''''''
+
+The proposals in this category suggest a new kind of declaration
+statement in the inner scope that prevents a name from becoming
+local. This statement would be similar in nature to the ``global``
+statement, but instead of making the name refer to a binding in the
+top module-level scope, it would make the name refer to the binding
+in the nearest enclosing scope.
+
+This approach is attractive due to its parallel with a familiar
+Python construct, and because it retains context-independence for
+function definitions.
+
+This approach also has advantages from a security and debugging
+perspective. The resulting Python would not only match the
+functionality of other nested-scope languages but would do so with a
+syntax that is arguably even better for defensive programming. In
+most other languages, a declaration contracts the scope of an
+existing name, so inadvertently omitting the declaration could yield
+farther-reaching (i.e. more dangerous) effects than expected. In
+Python with this proposal, the extra effort of adding the declaration
+is aligned with the increased risk of non-local effects (i.e. the
+path of least resistance is the safer path).
+
+Many spellings have been suggested for such a declaration:
+
+ - ``scoped x`` [1]_
+ - ``global x in f`` [3]_ (explicitly specify which scope)
+ - ``free x`` [5]_
+ - ``outer x`` [6]_
+ - ``use x`` [9]_
+ - ``global x`` [10]_ (change the meaning of ``global``)
+ - ``nonlocal x`` [11]_
+ - ``global x outer`` [18]_
+ - ``global in x`` [18]_
+ - ``not global x`` [18]_
+ - ``extern x`` [20]_
+ - ``ref x`` [22]_
+ - ``refer x`` [22]_
+ - ``share x`` [22]_
+ - ``sharing x`` [22]_
+ - ``common x`` [22]_
+ - ``using x`` [22]_
+ - ``borrow x`` [22]_
+ - ``reuse x`` [23]_
+
+The most commonly discussed choices appear to be ``outer``,
+``global``, and ``nonlocal``. ``outer`` is already used as both a
+variable name and an attribute name in the standard library. The
+word ``global`` has a conflicting meaning, because "global variable"
+is generally understood to mean a variable with top-level scope [24]_.
+In C, the keyword ``extern`` means that a name refers to a variable
+in a different compilation unit. While ``nonlocal`` is a bit long
+and less pleasant-sounding than some of the other options, it does
+have precisely the correct meaning: it declares a name not local.
+
+
+Proposed Solution
+=================
+
+The solution proposed by this PEP is to add a scope override
+declaration in the referring (inner) scope. Guido has expressed a
+preference for this category of solution on Python-Dev [14]_ and has
+shown approval for ``nonlocal`` as the keyword [19]_.
+
+The proposed declaration::
+
+ nonlocal x
+
+prevents ``x`` from becoming a local name in the current scope. All
+occurrences of ``x`` in the current scope will refer to the ``x``
+bound in an outer enclosing scope. As with ``global``, multiple
+names are permitted::
+
+ nonlocal x, y, z
+
+If there is no pre-existing binding in an enclosing scope, the
+compiler raises a SyntaxError. (It may be a bit of a stretch to
+call this a syntax error, but so far SyntaxError is used for all
+compile-time errors, including, for example, __future__ import
+with an unknown feature name.) Guido has said that this kind of
+declaration in the absence of an outer binding should be considered
+an error [16]_.
+
+If a ``nonlocal`` declaration collides with the name of a formal
+parameter in the local scope, the compiler raises a SyntaxError.
+
+A shorthand form is also permitted, in which ``nonlocal`` is
+prepended to an assignment or augmented assignment::
+
+ nonlocal x = 3
+
+The above has exactly the same meaning as ``nonlocal x; x = 3``.
+(Guido supports a similar form of the ``global`` statement [26]_.)
+
+On the left side of the shorthand form, only identifiers are allowed,
+not target expressions like ``x[0]``. Otherwise, all forms of
+assignment are allowed. The proposed grammar of the ``nonlocal``
+statement is::
+
+ nonlocal_stmt ::=
+ "nonlocal" identifier ("," identifier)*
+ ["=" (target_list "=")+ expression_list]
+ | "nonlocal" identifier augop expression_list
+
+The rationale for allowing all these forms of assignment is that it
+simplifies understanding of the ``nonlocal`` statement. Separating
+the shorthand form into a declaration and an assignment is sufficient
+to understand what it means and whether it is valid.
+
+
+Backward Compatibility
+======================
+
+This PEP targets Python 3000, as suggested by Guido [19]_. However,
+others have noted that some options considered in this PEP may be
+small enough changes to be feasible in Python 2.x [27]_, in which
+case this PEP could possibly be moved to be a 2.x series PEP.
+
+As a (very rough) measure of the impact of introducing a new keyword,
+here is the number of times that some of the proposed keywords appear
+as identifiers in the standard library, according to a scan of the
+Python SVN repository on November 5, 2006::
+
+ nonlocal 0
+ use 2
+ using 3
+ reuse 4
+ free 8
+ outer 147
+
+``global`` appears 214 times as an existing keyword. As a measure
+of the impact of using ``global`` as the outer-scope keyword, there
+are 18 files in the standard library that would break as a result
+of such a change (because a function declares a variable ``global``
+before that variable has been introduced in the global scope)::
+
+ cgi.py
+ dummy_thread.py
+ mhlib.py
+ mimetypes.py
+ idlelib/PyShell.py
+ idlelib/run.py
+ msilib/__init__.py
+ test/inspect_fodder.py
+ test/test_compiler.py
+ test/test_decimal.py
+ test/test_descr.py
+ test/test_dummy_threading.py
+ test/test_fileinput.py
+ test/test_global.py (not counted: this tests the keyword itself)
+ test/test_grammar.py (not counted: this tests the keyword itself)
+ test/test_itertools.py
+ test/test_multifile.py
+ test/test_scope.py (not counted: this tests the keyword itself)
+ test/test_threaded_import.py
+ test/test_threadsignals.py
+ test/test_warnings.py
+
+
+References
+==========
+
+.. [1] Scoping (was Re: Lambda binding solved?) (Rafael Bracho)
+ http://www.python.org/search/hypermail/python-1994q1/0301.html
+
+.. [2] Extended Function syntax (Just van Rossum)
+ http://mail.python.org/pipermail/python-dev/2003-February/032764.html
+
+.. [3] Closure semantics (Guido van Rossum)
+ http://mail.python.org/pipermail/python-dev/2003-October/039214.html
+
+.. [4] Better Control of Nested Lexical Scopes (Almann T. Goo)
+ http://mail.python.org/pipermail/python-dev/2006-February/061568.html
+
+.. [5] PEP for Better Control of Nested Lexical Scopes (Jeremy Hylton)
+ http://mail.python.org/pipermail/python-dev/2006-February/061602.html
+
+.. [6] PEP for Better Control of Nested Lexical Scopes (Almann T. Goo)
+ http://mail.python.org/pipermail/python-dev/2006-February/061603.html
+
+.. [7] Using and binding relative names (Phillip J. Eby)
+ http://mail.python.org/pipermail/python-dev/2006-February/061636.html
+
+.. [8] Using and binding relative names (Steven Bethard)
+ http://mail.python.org/pipermail/python-dev/2006-February/061749.html
+
+.. [9] Lexical scoping in Python 3k (Ka-Ping Yee)
+ http://mail.python.org/pipermail/python-dev/2006-July/066862.html
+
+.. [10] Lexical scoping in Python 3k (Greg Ewing)
+ http://mail.python.org/pipermail/python-dev/2006-July/066889.html
+
+.. [11] Lexical scoping in Python 3k (Ka-Ping Yee)
+ http://mail.python.org/pipermail/python-dev/2006-July/066942.html
+
+.. [12] Lexical scoping in Python 3k (Guido van Rossum)
+ http://mail.python.org/pipermail/python-dev/2006-July/066950.html
+
+.. [13] Explicit Lexical Scoping (pre-PEP?) (Talin)
+ http://mail.python.org/pipermail/python-dev/2006-July/066978.html
+
+.. [14] Explicit Lexical Scoping (pre-PEP?) (Guido van Rossum)
+ http://mail.python.org/pipermail/python-dev/2006-July/066991.html
+
+.. [15] Explicit Lexical Scoping (pre-PEP?) (Guido van Rossum)
+ http://mail.python.org/pipermail/python-dev/2006-July/066995.html
+
+.. [16] Lexical scoping in Python 3k (Guido van Rossum)
+ http://mail.python.org/pipermail/python-dev/2006-July/066968.html
+
+.. [17] Explicit Lexical Scoping (pre-PEP?) (Guido van Rossum)
+ http://mail.python.org/pipermail/python-dev/2006-July/067004.html
+
+.. [18] Explicit Lexical Scoping (pre-PEP?) (Andrew Clover)
+ http://mail.python.org/pipermail/python-dev/2006-July/067007.html
+
+.. [19] Explicit Lexical Scoping (pre-PEP?) (Guido van Rossum)
+ http://mail.python.org/pipermail/python-dev/2006-July/067067.html
+
+.. [20] Explicit Lexical Scoping (pre-PEP?) (Matthew Barnes)
+ http://mail.python.org/pipermail/python-dev/2006-July/067221.html
+
+.. [21] Sky pie: a "var" keyword (a thread started by Neil Toronto)
+ http://mail.python.org/pipermail/python-3000/2006-October/003968.html
+
+.. [22] Alternatives to 'outer' (Talin)
+ http://mail.python.org/pipermail/python-3000/2006-October/004021.html
+
+.. [23] Alternatives to 'outer' (Jim Jewett)
+ http://mail.python.org/pipermail/python-3000/2006-November/004153.html
+
+.. [24] Global variable (version 2006-11-01T01:23:16)
+ http://en.wikipedia.org/wiki/Global_variable
+
+.. [25] Ruby 2.0 block local variable
+ http://redhanded.hobix.com/inspect/ruby20BlockLocalVariable.html
+
+.. [26] Draft PEP for outer scopes (Guido van Rossum)
+ http://mail.python.org/pipermail/python-3000/2006-November/004166.html
+
+.. [27] Draft PEP for outer scopes (Nick Coghlan)
+ http://mail.python.org/pipermail/python-3000/2006-November/004237.html
+
+Acknowledgements
+================
+
+The ideas and proposals mentioned in this PEP are gleaned from
+countless Python-Dev postings. Thanks to Jim Jewett, Mike Orr,
+Jason Orendorff, and Christian Tanzer for suggesting specific
+edits to this PEP.
+
+
+Copyright
+=========
+
+This document has been placed in the public domain.
+
+
+..
+ Local Variables:
+ mode: indented-text
+ indent-tabs-mode: nil
+ sentence-end-double-space: t
+ fill-column: 70
+ coding: utf-8
+ End:
More information about the Python-checkins
mailing list