[Python-ideas] Tweaking closures and lexical scoping to include the function being defined

Nick Coghlan ncoghlan at gmail.com
Tue Sep 27 11:35:05 CEST 2011


On Tue, Sep 27, 2011 at 1:15 AM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Nick Coghlan wrote:
>
>> Now, the use of 'nonlocal' in the inner function has *changed the
>> lifecycle* of x. It is now a nonlocal variable - it survives beyond
>> the execution of the function that defines it.
>
> Yes, but that's just a logical consequence of the fact
> that it's *visible* from the inner function, together
> with the fact that Python never throws anything away
> while it's still accessible. Visibility is still the
> primary concept attached to 'nonlocal'.

Hmm, I think you just summarised the key insight that led to me
suggesting nonlocal in the first place (even though I wasn't thinking
of it in exactly these terms at the time).

A local name binding is visible only from the current *invocation* of
a function, hence it only lasts until the call returns.

On the other hand, an explicitly 'nonlocal' name binding is visible from:
- the current invocation of the outer function where the name is defined
- all invocations of any functions defined within that outer function

As you note, the changes in lifecycle then follow from the change in
visibility - as long as any of those inner functions survive, so does
the name binding. Non-local name bindings can of course also happen
implicitly if there's no local assignment to override.

My proposal is merely that we add a *new* meaning to 'nonlocal' that
supplements the existing meaning, to declare that a name is visible
from all invocations of the function currently being defined rather
than being local to each invocation.

As with existing nonlocal usage, the change in visibility (all
invocations vs current invocation) then leads directly to a change in
lifecycle (linked to the lifetime of the function object rather than
until an individual call returns). The 'nonlocal' terminology could
also be seen as referring to the evaluation scope for the
initialisation expression, since it would run in the surrounding
scope, just like default argument values.

Since the new semantics slots in neatly between actual locals and the
current usage of 'nonlocal', and because 'nonlocal' itself is such a
generic term, it just seems easier to me to teach people about the
link between visibility and lifetime and two possible meanings for
nonlocal than it would be to:
    1. find a new term (we've been trying and failing at this for years now)
    2. teach people what it means

Step 2 doesn't get any easier just because we use a different name -
if anything it gets harder, since people are already used to nonlocal
as a modifier on name visibility, and the new term will need to
indicate a closely related concept.

Is giving 'nonlocal' a second related-but-not-identical meaning a
perfect answer? No, I don't think so. However, when generators were
added, the 'def' keyword went from unambiguously declaring a function
to declaring either an ordinary function or a generator-iterator
factory and people seemed to cope. 'yield' and 'yield from' don't mean
exactly the same thing, but I expect to be able to cope with that as
well.

If someone can understand the concept of function scoped variables at
all, then they'll be able to understand the difference between
'nonlocal VAR' and 'nonlocal VAR from EXPR'. Given past history, I
seriously doubt our ability to come up with a keyword for function
scoped variables that is any more intuitive than a variation on
nonlocal.

We would then have the following hierarchy of visibility:

Local scope (VAR = EXPR): visible from current function invocation
Function scope (nonlocal VAR from EXPR): visible from all invocations
of this function
Lexical scope (nonlocal VAR; VAR = EXPR): visible from outer function
where VAR is a local and all invocations of this and any peer
functions
Module scope (global VAR; VAR = EXPR): visible from all code in the
current module
Process scope (import builtins; builtins.VAR = EXPR): visible from all
code in the current process

For name *lookup*, lexical scoping, module scoping and process scoping
can all happen implicitly when a name doesn't refer to a local.
Function scope lookup would only happen with an explicit declaration
(regardless of how it was spelt).

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia



More information about the Python-ideas mailing list