[Python-ideas] Statement local functions and classes (aka PEP 3150 is dead, say 'Hi!' to PEP 403)

[Nick Coghlan]

On Thu, Oct 13, 2011 at 4:37 PM, Eric Snow <ericsnowcurrently at gmail.com> wrote:
> On Thu, Oct 13, 2011 at 12:30 AM, Carl M. Johnson
> <cmjohnson.mailinglist at gmail.com> wrote:
>>
>> On Oct 12, 2011, at 8:24 PM, Ka-Ping Yee wrote:
>>
>>> I don't see why the leading colon is necessary, actually.  Isn't
>>> the presence of a bare @ enough to signal that a function or class
>>> definition must come next?
>>
>> Python's parser is purposefully pretty simple, so it's not clear if that could be added without ripping up the whole language definition.
>
> and visually the colon _does_ make the syntax clearer.  It works
> because it's so out of place at the beginning of a line.  Scanning
> through code you won't miss it (and misinterpret what's going on).

Yeah, the block prefix needs something to mark it as special, because
it *is* special (even more so than decorators). Decorator expressions
are at least evaluated in the order they're encountered in the source
- it is only the resulting decorators that are saved and invoked later
(after the function has been defined).

These block prefix lines would be different, they'd only be evaluated
*after* the function was already defined and any decorators applied.

We could probably make the parsing unambiguous without a prefix syntax
if we really wanted to (especially if we used a different symbol to
reference the object being defined), but the out of order execution of
the associated statement makes that a questionable idea.

That's basically why I went with the leading ':' - it's jarring enough
to get your attention, without being so ugly as to be intolerable.
However, Georg's "this doesn't look like Python any more" criticism
has serious merit - while style guidelines can mitigate that to some
degree (just as they advise against gratuitous use of lambda
expressions when a named function would be better), there's an
inherent ugliness to the syntax in the first draft of the PEP that may
make it irredeemable.

I think PEP 403 in its current form is most useful as a statement of
intent of the kind of code we want to be able factor cleanly. The
decorator syntax handles code specifically of the form:

  def|class NAME HEADER:
      BODY
  NAME = decorator(NAME)

PEP 403 instead sets out to handle the case of:

  def NAME HEADER:
      BODY
  # operation involving NAME
  del NAME (unless the operation was an explicit assignment to NAME,
in which case we leave it alone)

The key point is that we don't really *care* about the function being
defined - we really only care about the operation we need it for (such
as using it as a callback or as a key function or returning it or
yielding it or even calling it and doing something with the result).
It's essentially a one shot operation - it may be *invoked* more than
once, but we're defining it for the purposes of getting someone else
to run a chunk of our code, *not* for the purpose of explicitly
invoking that operation in multiple places in the application's source
code.

Hopefully at this point those that have wished for and argued in
favour of multi-line lambdas over the years are nodding their heads in
agreement - I think I finally get what they've been complaining about
for so long, and PEP 403 is the result. If it doesn't address at least
a significant fraction of their use cases, then we need to know.

I already think there are two simplifying assumptions that should be
made at least for the first iteration of the idea:

1. Leave classes out of it, at least for now. We did that with
decorators, and I think it's a reasonable approach to follow.
2. The initial version should be an alternative to decorator syntax,
not an addition to it. That is, you wouldn't be able to mix the first
incarnation of a block prefix with ordinary decorators.

If we can find an approach that works for the basic case (i.e. naked
function definition) and doesn't make people recoil in horror (esp.
Guido), then we can look at expanding back to cover these two cases.

I'll note that the only further syntax idea I've had is to replace the
':' with 'postdef' and the '@' with 'def', so some of the examples
kicking around would become:

    postdef x = weakref.ref(obj, def)
    def report_destruction(obj):
        print("{} is being destroyed".format(obj))

    postdef funcs = [def(i) for i in range(10)]
    def make_incrementor(i):
        postdef return def
        def incrementor(x):
            return x + i

    postdef sorted_list = sorted(original, key=def)
    def normalise(item):
        …

That actually looks quite readable to me, and is fairly explicit about
what it does: here's a piece of code to run after the following
function has been defined. I definitely like it better than what I
have in the PEP.

With this variant, I would suggest that any postdef clause be executed
*in addition* to the normal name binding. Colliding on dummy names
like "func" would then be like colliding on loop variables like "i" -
typically harmless, because you don't use the names outside the
constructs that define them anyway.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia