[issue26205] Inconsistency concerning nested scopes

New submission from Roscoe R.Higgins: In chapter 9. Classes of the Python3.5 documentation it says: "At any time during execution, there are at least three nested scopes whose namespaces are directly accessible:", followed by a list containing 4 items. Further down a middle scope is mentioned (although mentioned by name). This was confusing for a while. ---------- assignee: docs@python components: Documentation messages: 258941 nosy: Roscoe R. Higgins, docs@python priority: normal severity: normal status: open title: Inconsistency concerning nested scopes type: behavior versions: Python 3.5 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue26205> _______________________________________

Brett Cannon added the comment: https://docs.python.org/3/tutorial/classes.html for the docs that Roscoe is talking about. So the sentence is technically correct, it just takes careful reading to grasp what's being said. There are "at least three nested scopes", but there can be *up to* four scopes. Since "the scopes of any enclosing functions" is not necessarily true for all code scopes, you end up with at least three, but possibly four scopes. Obviously the wording could be clearer, so if you want to sign the CLA, Roscoe, and propose a rewording that would be appreciated! ---------- nosy: +brett.cannon _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue26205> _______________________________________

Yannick Duchêne added the comment: Better at least two, if I'm not wrong: the innermost scope may be the module's scope. So there is always at least the module scope and the built‑ins scope, at least two up to four. (if I have not overlooked something) ---------- nosy: +Hibou57 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue26205> _______________________________________

Brett Cannon added the comment: It depends on how you want to view things as to whether you can claim there are two or three scopes for module-level code. While it's true that the local scope operates just like global scope, to Python it's more like local == global rather than the local scope simply doesn't exist (hence why `locals()` never throws an exception saying the scope doesn't exist but instead is the same as `globals()`). The "three scope" phrasing also predates nested scopes from back when Python had its LGB scoping rules: Local-Global-Builtin. Back then we just said Python had three scopes and left it at that since it was basically always true and didn't confuse anyone. At this point I'm fine with just removing the number from the sentence and saying something like "At any time during execution, there are various nested scopes whose namespaces are directly accessible:". ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue26205> _______________________________________

Changes by Brett Cannon <brett@python.org>: ---------- stage: -> needs patch type: behavior -> versions: +Python 2.7, Python 3.6 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue26205> _______________________________________

Terry J. Reedy added the comment: Would 'three or more' be any clearer than 'at least three'? They mean the same, but the first seems better to me in this context. The real problem with this section are a) the use of Guido's first person 'I' and b) statements that were not changed when nested scope were added, but should have been. "If a name is declared global, then all references and assignments go directly to the middle scope containing the module’s global names." The global scope is no longer the middle scope. Roscoe pointed at this. With that removed, the sentence says that if a name is declared global, assignments go to global scope. This would be more meaningful if prefixed by a revised version of the following, which is several paragraphs down. "A special quirk of Python is that – if no global statement is in effect – assignments to names always go into the innermost scope." The special quirk part should go; 'global' would now have to be 'global or nonlocal', but I now think the following, preceeding the revised 'global' sentence above, would be better. "By default, assignments to names always go into the innermost, local, namespace." In other words, I think the sentence Roscoe flagged is the least of the problems with this section. ---------- nosy: +terry.reedy _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue26205> _______________________________________

Changes by Ezio Melotti <ezio.melotti@gmail.com>: ---------- nosy: +ezio.melotti type: -> enhancement _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue26205> _______________________________________

Cheryl Sabella <cheryl.sabella@gmail.com> added the comment: Assigning to @Mariatta for the sprints. ---------- assignee: docs@python -> Mariatta nosy: +Mariatta, cheryl.sabella versions: +Python 3.7, Python 3.8 -Python 2.7, Python 3.5, Python 3.6 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue26205> _______________________________________

Change by Joannah Nanjekye <nanjekyejoannah@gmail.com>: ---------- keywords: +patch nosy: +nanjekyejoannah nosy_count: 8.0 -> 9.0 pull_requests: +20474 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/21324 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue26205> _______________________________________

Vedran Čačić <vedgar@gmail.com> added the comment: Just to point out: it can be more than four, right? If you have a function aa within a function bb within a function cc, then bb's and cc's scope are different, even though functionally equivalent from the perspective of aa. ---------- nosy: +veky _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue26205> _______________________________________

Terry J. Reedy <tjreedy@udel.edu> added the comment: At any one time, the language and interpreter sees the local scopes of enclosing functions as collectively 'nonlocal' for the purpose of accessing and rebinding. If there are bindings of a name 'x' in multiple enclosing local scopes, the binding for 'x' in the synthesized 'nonlocal' is the innermost one. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue26205> _______________________________________

Vedran Čačić <vedgar@gmail.com> added the comment: How is that different from saying that "At any one time, the language and interpreter sees all the scopes as one scope for the purpose of accessing and rebinding."? Any ChainMap presents itself as a dictionary to the outside. That doesn't mean it doesn't have an internal structure. Yes, nonlocal picks the innermost one because of its semantics, but it still needs to search through the scopes in order to know which one to pick, right? ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue26205> _______________________________________

Terry J. Reedy <tjreedy@udel.edu> added the comment: Vedran: "How is that different from saying that "At any one time, the language and interpreter sees all the scopes as one scope for the purpose of accessing and rebinding."?" For access, one may bypass the default staged access by specifying a specific namespace. For rebinding elsewhere than in locals, one must specify one of the others. So the 4 scopes are each visible. However, the internal 'structure' of nonlocals is intentionally not visible or accessible. (The proposal to make them somehow accessible was considered and rejected.) --- The PR replaces 'at least three' with 'various'. This is needlessly vague. There are always 3 namespaces and sometimes a 4th, so I think 'at least three' should be replaced with the specific '3 or 4'. I rechecked two special cases. Exception clauses with 'as name' bind name in the current local namespace and auto delete the binding when done. Comprehensions bind the name and iterator of the 'for' clause in a temporary new local namespace, making the existing namespace temporarily part of a nonlocal namespace, which may or may not have been present before. '3 or 4' still applies in both cases. ---------- versions: +Python 3.10 -Python 3.7, Python 3.8 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue26205> _______________________________________

Joannah Nanjekye <nanjekyejoannah@gmail.com> added the comment: PR updated with your change. @Terry ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue26205> _______________________________________

Change by Terry J. Reedy <tjreedy@udel.edu>: ---------- title: Inconsistency concerning nested scopes -> Better specify number of nested scopes _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue26205> _______________________________________

Joannah Nanjekye <nanjekyejoannah@gmail.com> added the comment: New changeset 9ed3cd8ba052b395ab50692bb65988b065d68e27 by Joannah Nanjekye in branch 'master': bpo-26205: Specify the number of nested scopes (GH-21324) https://github.com/python/cpython/commit/9ed3cd8ba052b395ab50692bb65988b065d... ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue26205> _______________________________________

Change by miss-islington <mariatta.wijaya+miss-islington@gmail.com>: ---------- nosy: +miss-islington nosy_count: 10.0 -> 11.0 pull_requests: +20489 pull_request: https://github.com/python/cpython/pull/21341 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue26205> _______________________________________

Change by miss-islington <mariatta.wijaya+miss-islington@gmail.com>: ---------- pull_requests: +20490 pull_request: https://github.com/python/cpython/pull/21342 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue26205> _______________________________________

Joannah Nanjekye <nanjekyejoannah@gmail.com> added the comment: Merged, this can be closed if there is consesus. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue26205> _______________________________________

Joannah Nanjekye <nanjekyejoannah@gmail.com> added the comment: New changeset 7ceb3e3ffc8ee00551df2245544eb60f7debf3af by Miss Islington (bot) in branch '3.8': bpo-26205: Specify the number of nested scopes (GH-21324) (GH-21342) https://github.com/python/cpython/commit/7ceb3e3ffc8ee00551df2245544eb60f7de... ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue26205> _______________________________________

Terry J. Reedy <tjreedy@udel.edu> added the comment: I wait for backports to be merged before closing. Someone has to approve them or manually merge them before that happens, so leaving issues open until all merges are done prevents totally forgetting about them. For doc issues, the patch, if not drastically wrong, becomes the new status quo. So I think any further proposed change should normally be a new issue. So once the backports *are* done, you can close. For code issues, 'drastically wrong' usually means that the patch failed on one of the buildbots, or fairly soon somewhere else, and needs either reversion or a hot fix. ---------- versions: +Python 3.8, Python 3.9 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue26205> _______________________________________

Joannah Nanjekye <nanjekyejoannah@gmail.com> added the comment: New changeset 3f4a9fd912fc6d4f5ee2b49bfef979cc7d457848 by Miss Islington (bot) in branch '3.9': bpo-26205: Specify the number of nested scopes (GH-21324) (GH-21341) https://github.com/python/cpython/commit/3f4a9fd912fc6d4f5ee2b49bfef979cc7d4... ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue26205> _______________________________________

Vedran Čačić <vedgar@gmail.com> added the comment: Well, then "what name is in which namespace" is relative to which function we're considering. In my example above, imagine aa has x declared, bb has y, and cc has z. Then y and z are in the same namespace when we look from the perspective of aa, but they are not in the same namespace from the perspective of bb. Even worse, cc sees z but doesn't see y. How can they be in the same namespace then? I always thought about namespaces as mappings from names to objects, independent of perspective. Whether two names are in the same namespace, should be a question with an objective answer. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue26205> _______________________________________

Terry J. Reedy <tjreedy@udel.edu> added the comment: Since you ask, here is a extended summary of namespaces. There is one built-in namespace. There is one global namespace for each module, which is also the local namespace for top level code in that module. There is one local namespace for each class and function. There is one nonlocal namespace for each nested function. "At any time during execution" (the beginning of the doc sentence targeted by this issue), there are 3 or maybe 4 namespaces accessible with simple undotted names. Module global and class local namespaces, including nested classes, are accessible from without the object with dotted names. There are rules for class and method code about access to superclass namespaces. Function locals are not necessarily accessible with standard python code, but at least in cpython, local names can be accessed via code objects. Default parameter values and current nonlocal values, when set, can be accessed via function objects. Dotted names are discussed elsewhere in the tutorial, while function and code object introspection is out of scope there. (And off topic for this issue ;-). ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue26205> _______________________________________

Terry J. Reedy <tjreedy@udel.edu> added the comment: Besides the sentence now revised, the initial post referenced confusion with 'middle scope' in "If a name is declared global, then all references and assignments go directly to the middle scope containing the module's global names." This has not been directly discussed that I could find, but it seems inappropriate both at toplevel, where globals = locals, and in nested functions, where nonlocals are also 'middle'. Maybe replace "the middle scope containing the module's global names" with "the module's global namespace". The rest of this paragraph and the next could be reviewed. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue26205> _______________________________________

Eryk Sun <eryksun@gmail.com> added the comment:
There is one local namespace for each class and function. There is one nonlocal namespace for each nested function.
In the first sentence, replace "class" with "class statement" and "function" with "function call". The second sentence could use "nested-function call", but maybe rewrite it as, "There is one nonlocal namespace when a nested function is called." ---------- nosy: +eryksun _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue26205> _______________________________________

Change by Brett Cannon <brett@python.org>: ---------- nosy: -brett.cannon _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue26205> _______________________________________
participants (10)
-
Brett Cannon
-
Cheryl Sabella
-
Eryk Sun
-
Ezio Melotti
-
Joannah Nanjekye
-
miss-islington
-
Roscoe R.Higgins
-
Terry J. Reedy
-
Vedran Čačić
-
Yannick Duchêne