[Python-ideas] A comprehension scope issue in PEP 572

[Tim Peters]

[Guido]
> I am convinced by Tim's motivation. I hadn't thought of this use case before
> -- I had mostly thought "local scope in a comprehension or generator
> expression is the locals of the synthetic function". But Tim's reasoning
> feels right.

I'm trying very hard _not_ to reason.  That is, I'm looking at code
and trying to figure out what would actually work well, logic &
consistency & preconceptions be damned.  "Reasons" can be made up
after the fact - which is how the world actually works regardless ;-)


> The only solution that makes sense to me is Steven's (2). (1) is what the
> PEP currently says and what Tim doesn't want; (3) has no precedent (function
> defaults don't really work like this) and just gets my hackles all up. (I
> can't even tell if Steven meant it as a serious proposal.)

He doesn't want (3) either.  I can channel him on that.


> So let's see if we can change PEP 572 so that := inside a comprehension or
> generator expression always assigns to a variable in the containing scope.

While I don't have real use cases beyond that, given that much,
"consistency" kicks in to suggest that:

    def f():
         [x := 42 for x in range(1)]

makes `x` local to `f` despite that x wasn't bound elsewhere in f's body.

    def f():
        global x
         [x := 42 for x in range(1)]

binds the global `x`.

    def f():
        nonlocal x
         [x := 42 for x in range(1)]

binds `x` in the closest-containing scope in which `x` is local.  The
last two act as if the declaration of `x` in `f` were duplicated at
the start of the synthetic function.

More succinctly, that `x := RHS` in a synthetic function "act the
same" as `x = RHS` appearing in the scope directly containing the
synthetic function.

Does that generalize to class scope too?  I don't know.  I never use
fancy stuff in class scopes, and have no idea how they work anymore.
So long as "def method(self, ...)" continues to work, I'm happy ;-)


> It may be inconsistent with the scope of the loop control variable, but it's
> consistent with uses of := outside comprehensions:
>
>   [x := 0]
>   [x := i for i in range(1)]
>
> both have the side effect of setting x to zero. I like that.

And is what anyone would expect if they didn't think too much about it.

... [snipping stuff about class scope - nothing to add] ...


> PS1. The various proposals that add random extra keywords to the syntax
> (like 'for nonlocal i') don't appeal to me at all.

They're appealing to the extent that "explicit is better than
implicit" for people who actually understand how all this stuff is
implemented.  I don't know what percentage of Python programmers that
is, though.  I've certainly, e.g., seen many on Stackoverflow who can
make a list comprehension work who couldn't distinguish a lexical
scope from an avocado.  The ":= is treated specially in
comprehensions" idea is aimed more at them than at people who think
invoking a synthetic anonymous lambda "is obvious".


> PS2. IIRC the reason we gave loop control variables their own scope was the
> poor imagination of many people when it comes to choosing loop control
> variable names. We had seen just too many examples of
>
>   for x in something:
>       ...lots of code using x...
>       blah blah [x+1 for x in something else]
>       ...some more code using x, broken...
>
> It's true that this can also happen with a for-loop statement nested inside
> the outer loop (and it does) but the case of a comprehension was easier to
> miss. I've never looked back.

I don't want to change anything about any of that - already believe
Python struck the best balance possible.


> PS3. Comprehensions and generator expressions should be interchangeable.
> They just look too similar to have different semantics (and the possibly
> delayed execution of generator expressions is not an issue -- it's rare, and
> closure semantics make it work just fine).

Wholly agreed.