[Python-ideas] Tweaking closures and lexical scoping to include the function being defined

[Eric Snow]

On Tue, Sep 27, 2011 at 2:15 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> On 27 September 2011 20:53, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> I actually changed my mind on that front when replying to Greg - I now
>> agree that visibility is the important aspect, but I also think there
>> are two levels of visibility within a function: local scope, which is
>> distinct for every call to the function, and "function scope", which
>> is shared across all invocations to the function.
>
> That description just makes my head hurt. To me, visibility is simply
> which lines of code, when using a name, refer to that particular
> variable. So a variable x at module level has "global scope" because
> it is visible from all lines (in the module, ignoring shadowing
> concerns). If x is defined within a function, it has local (to the
> function) scope, because only lines within the function can see that
> variable. In other words scope is essentially a lexical concept tied
> to lexical visibility.

Perhaps that is why people sometimes miss that default argument values
persist across calls.  The names are locals, but the values live in an
often unconsidered (and relatively artificial) "function definition
scope".

The variables in the proposed nonlocal-from statement are in that same
function definition scope implicitly.  However, unlike other names,
they're bound to their scope by a statement in a different scope (the
local one).

Also, since they are variables they would be stored as cells (like
__closure__ does).  This is another contrast to default arguments,
which are stored only as values (in __defaults__ and __kwdefaults__).

The function definition scope isn't the only source of confusion here.
 In one context "nonlocal" would mean "not local (and not global)".
In another context it would mean "in function definition scope".

Here's a different angle to show what I mean (I'm not proposing this).
 Currently, if a nonlocal statement refers to a name that is not bound
in an enclosing function scope, it gives a syntax error:

  >> def f():
  ...     def g():
  ...        nonlocal x
  ...     return g
  ...
  SyntaxError: no binding for nonlocal 'x' found

If that situation instead implied "function definition scope", you
would get a similar effect to nonlocal-from: persistent state for
select variables across calls.  However, the behavior would be
consistent across all uses of nonlocal.  "nonlocal x=5" (if allowed)
would be the same as "nonlocal x; x=5".  You'd lose the ability to set
a one-time initial value, but the point is that nonlocal would become
the means to interact with variables in the function definition scope.

Ultimately, it seems like we are looking for a way to do three things here:

1. bless a concrete function definition scope (persist across calls),
2. allow one-time initialization of those persistent variables,
3. (relatedly) have some expressions evaluated at definition time but
available at execution time.

Default arguments cover some of this.  Closures cover some of it.  The
nonlocal-from statement covers a bunch.

-eric

>
> To that extent, these new closed-over values have local scope because
> they are visible from the same lines of code as a local variable. But
> their lifetime differs.
>
> The ideas here are more commonly seen in the history of Lisp, which
> originally had "dynamic" scope (which is more lifetime-oriented) but
> moved to lexical (visibility-based) scope. There was a time when Lisps
> were described as Lisp-1 or Lisp-2 depending on their position on
> scope.
>
> I need to go and do some research into old Lisp concepts to see how
> they might apply here. I may be some time :-)
>
> Paul.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>