[Python-ideas] Tweaking closures and lexical scoping to include the function being defined

[Nick Coghlan]

On Wed, Sep 28, 2011 at 3:10 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> Between these points and Arnaud Delobelle's point that code inside a
> function should do nothing when the def itself is executed, I'm
> getting more convinced that objects with persistent local scope should
> be introduced *outside* the function body.
>
> That either validates the default-argument hack as a valid response to
> a specific requirement, or suggests some syntax added to the function
> definition line. Or of course, just go with a normal (nested function)
> closure.

I'll point out that there's one thing the default argument hack
*doesn't* do that is relevant to the discussion: because it doesn't
actually move the name into a separate scope (only the value), it has
no effect on rebinding. It's a lot like closures in the days before
nonlocal declarations:

    def accumulator():
        def incr(x, *, _tally=0):
            _tally += x
            return _tally
        return incr

That doesn't actually work, since _tally is an ordinary local that
gets reinitialised to zero on each invocation. The only syntax
suggested so far that meets the criteria of being both in the function
header and creating a separate namespace for the namebindings is the
one I put forward in Ron's previous thread (I've excluded the
'after-**' syntax here, since this thread has convinced me it would be
crazy to use something like that when the name binding semantics are
so different from those of a default argument):

    def accumulator():
        def incr(x) [tally=0]:
            tally += x
            return tally
        return incr

As the rough equivalent of today's:

    def accumulator():
        tally = 0
        def incr(x):
            nonlocal tally
            tally += x
            return tally
        return incr

The objections to that proposal that I recall were:
1. Can't look up [] in the help index (Agreed, but you can look up
'def' which is where these function state variables would be
documented)
2. Noise in the header line (Agreed, but no worse than the default
argument hack)
3. The behaviour is very different from a list (Agreed, but function
parameters and arguments are very different from tuples and people can
cope with that)
4. It should be in the function body (I think Arnaud and Paul have
raised some very good points in this thread for why it *shouldn't* be
in the function body)

As far as the 'hidden state' problem goes, I'd prefer to address that
by providing better utilities in functools for accessing closure state
(and perhaps even the current internal state of suspended generators).
As I said in the previous thread, the "algorithm with shared state"
model represented by closures (and, with a slightly different spin on
the concept, generators) and the "data with behaviour" model
represented by object-oriented programming are very different ways of
framing a problem and forcing people to switch models because of
internal language issues unrelated to the real world problem they're
trying to describe isn't a good thing. I agree closures have
testability problems, but I don't think telling people "avoid closures
because they have testability problems" is the right answer to that.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia