[Python-Dev] Looking for examples: proof that a list comp is a function

[Tim Peters]

[ Tim, about the most version of the docs at
    https://docs.python.org/dev/reference/expressions.html#displays-for-lists-sets-and-dictionaries
]
>> I say "pretty much" because, for whatever reason(s), it seems to be
>> trying hard _not_ to use the word "function".  But I can't guess what
>> "then passed as an argument to the implicitly nested scope" could
>> possibly mean otherwise (it doesn't make literal sense to "pass an
>> argument" to "a scope").

[Nick Coghlan <ncoghlan at gmail.com>]
> I think my motivation was to avoid promising *exact* equivalence with a
> regular nested function, since the define-and-call may allow us
> opportunities for optimization that don't exist when those two are separated
> (e.g. Guido's point in another thread that we actually avoid calling "iter"
> twice even though the nominal expansion implies that we should). However,
> you're right that just calling it a function may be clearer than relying on
> the ill-defined phrase "implicitly nested scope".

Plus that, as noted, what passing an argument "to a scope" means is mysterious.

Language standard committees struggle for years with how to phrase
things so that no more than is intended appears to be promised.  It's
hard!  For example, if you were to show a workalike function and note
that the exact placement - and number - of `iter()` calls is not
guaranteed, someone else would point out that you need to explicitly
say that by "iter" you mean the builtin function of that name, not one
user code may have overridden it with in the current scope.   Then
someone else will note that it's tedious to say things like that
whenever they're needed, and more-general text will be added elsewhere
in the docs saying that the _rest_ of the docs always mean the
language-supplied versions of such-&-such explicitly named
functions/classes/modules/...

I'd say "nested function" anyway ;-)  And for another reason:  not
just someone from Mars is prone to misreading 'scope", but just about
anyone on Earth coming from another language.  The idea that the word
"scope" all by itself implies "and in general any name bound to within
the top-level code spanned by the scope is implicitly local to the
scope unless explicitly declared `global` or `nonlocal` in the scope"
may be unique to Python.


> For Chris's actual question, this is part of why I think adding
> "parentlocal" would actually make the scoping proposal easier to explain, as
> it means the name binding semantics aren't a uniquely magical property of
> binding expressions (however spelled), they're just a new form of target
> scope declaration that the compiler understands, and the binding expression
> form implies. Note: eas*ier*, not easy ;)

Adding an explanation of `parentlocal` to the docs could be a useful
pedagogical device, but I don't think I'd support adding that
statement to the _language_.  It's too weird, and seems to be at a
wrong level for plausible future language developments.

Let's step waaaaay back for a minute.  In many languages with
full-blown closures, first-class functions, and nested lexical scopes,
it's pretty common to define the meaning of various language
constructs in terms of calling derived lexically nested functions.  In
those languages, any "work variables" needed by the synthetic
functions are declared as being local to those functions, and _that's
the end of it_.  They're done.  All other names inside the expansions
mean exactly the same as what they mean in whatever chunks of
user-supplied code the construct interpolates into the synthesized
functions.  It doesn't matter one whit in which context(s) they
appear.

That's the only long-term sane way to go about defining constructs in
terms of calling synthesized functions interpolating user-supplied
pieces of code.

Now _if_ Python had been able to do that, the meaning of genexps and
listcomps would have been defined, from the start, in terms of
synthesized functions that declared all & only the for-target names
"local".  And, in fact, the change I'm suggesting wouldn't have
required changing the comprehension implementation _at all_ when
assignment expressions were added.  Instead the implementation would
need to change to _add_ assignment expression targets to the things
declared local if it was decided that those targets should be _local_
to the derived functions instead.

That's why this all seems so bloody obvious to me ;-)  It's how
virtually every other language in the business of defining constructs
in terms of nested synthesized functions works.

So if that's something we may ever do again - and possibly even if we
don't expect to ever do it again - I suggest a more generally useful
approach would be to add a new flavor of _function_ to Python.  Namely
one wherein the only locals are the formal arguments and those
explicitly declared local.  Whether or not a name is bound in the body
would be irrelevant.  To avoid a new keyword, `local` could be spelled
`not nonlocal` ;-)

Note that the only use for `parentlocal` so far is tediously emulating
the _effects_ of what that hypothetical `deflocal` flavor of function
would do all the time with names not declared local in it.  If we ever
do something like this again, it would be far easier and clearer to
just say the synthetic functions are of the `deflocal` flavor, and
here are the names declared local in this case:  x, y, z, ...

User-defined functions may well find that useful at times too.
Although it would be a large conceptual addition to part of Python,
adding `parentlocal` would be too, and all by itself the latter looks
like an incoherent pile of bizarre tricks.  The _meaning_ of
`deflocal` would be immediately clear to people coming from any number
of other modern-ish languages.


> It also occurs to me that we could do something pretty neat for class
> scopes: have parent local declarations in methods target the implicit
> lexical scope where __class__ lives (to support zero-arg super), *not* the
> class body. That would entail adding a "classlocal" declaration to target
> that implied scope, though.
>
> That would give the following definition for "lexical scopes that parent
> local scoping can target":
>
> - module globals (parentlocal -> global)
> - function locals, including lambda expression locals (parentlocal ->
> nonlocal)

Except that for top-level functions, parentlocal -> global; and
regardless of nesting level, also implies `global` if the name is
declared `global` in the parent block.  Unless I've wholly lost track
of your intent, which is quite possible.

=== no new content below ===

> - implicit class closure, where __class__ lives (parentlocal -> nonlocal in
> current scope, classlocal in class scope)
>
> Most notably, in the synthetic functions created for generator expressions
> and comprehensions, a parentlocal declaration in a child scope would imply a
> parentlocal declaration in the synthetic function as well, propagating back
> up the chain of nested lexical scopes until it terminated in one of the
> above three permitted targets.
>
> Using the explicit forms would then look like:
>
>     from __future import parent_scopes # Enable the explicit declaration
> forms
>
>     class C:
>         classlocal _n # Declares _n as a cell akin to __class__ rather than
> a class attribute
>         _n = []
>         @staticmethod
>         def get_count():
>             return len(_n)
>
>     assert not hasattr(C, "_n")
>     assert C.get_count() == 0
>
>     def  _writes_to_parent_scope():
>         parentlocal outer_name
>         outer_name = 42
>
>    assert outer_name == 42
>
> I'm still doubtful the complexity of actually doing that is warranted, but
> I'm now satisfied the semantics can be well specified in a way that allows
> us to retain the explanation of generator expressions and comprehensions in
> terms of their statement level counterparts (with the added bonus of making
> "__class__" a little less of a magically unique snowflake along the way).