[Python-ideas] Tweaking closures and lexical scoping to include the function being defined

[Guido van Rossum]

On Wed, Sep 28, 2011 at 3:45 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Actually, there are some additional aspects that annoy me:
> - the repetition of the variable name 'n'
> - the scope of the variable name is wrong (since we only need it in
> the inner function)
> - indentation matters (cf try/except/finally)
> - hidden signature for the actual function
> - hoops to jump through to get decent introspection values (e.g. __name__)
>
> I find the following 5 line toy example:
>
> from threading import Lock
>
> def global_counter() [n=1, lock=Lock()]:
>    with lock:
>        print(n)
>        n += 1

Hm, this syntax is *too* concise. It's lack of keywords makes it open
to misinterpretation.

> Far more readable than the 10-line closure equivalent:
>
> from threading import Lock
>
> @apply # I sometimes wonder if we should bring this back in functools...
> def global_counter():
>    n = 1
>    local = Lock()
>    def global_counter():
>        with lock:
>            print(n)
>            n += 1
>    return global_counter

True, the outer function completely obscures what's going on.

> Or the 10-line 7-self class equivalent:
>
> from threading import Lock
>
> @apply
> class global_counter:
>    def __init__(self):
>        self.n = 1
>        self.lock = Lock()
>
>    def __call__(self):
>        with self.lock:
>            print(self.n)
>            self.n += 1

Yeah, anything involving __call__ is hard to figure out.

So after reading some previous messages, I almost wish we could
implement the "decorator + nonlocal statement" proposal:

@init(n=1, lock=Lock())
def global_counter():
  nonlocal n, lock
  with lock:
    print(n)
    n += 1

Alas, the problem is that you can't actually implement such a
decorator, not even using an extension module, because the compiler,
upon seeing the "nonlocal n, lock", ignoring the decorator (since it
should be applied at run time, not at compile time), will complain
that there isn't actually an intermediary outer scope defining either
n or lock.

Some ways out of this:

- let the compiler special-case a certain built-in decorator name at
compile-time (unorthodox, not impossible)

- different syntax, e.g.
  @[n=1, lock=Lock()]
or
  @in(n=1, lock=Lock())
or even
  in n=1, lock=Lock()

(all followed by "def global_counter() etc.")

Of course once there's different syntax, the nonlocal declaration in
the function is redundant. And clearly I'm back-peddling. :-)

-- 
--Guido van Rossum (python.org/~guido)