[Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Nick Coghlan]

On 24 June 2018 at 15:56, Steven D'Aprano <steve at pearwood.info> wrote:
> On Sun, Jun 24, 2018 at 02:33:59PM +1000, Nick Coghlan wrote:
>
>> Given that PEP 572 *is* proposing implicit comprehension state export,
>
> "Implicit" and "explicit" are two terms which often get misused to mean
> "I don't like it" and "I do like it".
>
> Making the intentional choice to use an assignment expression is not
> really "implicit" in any meaningful sense.

No, it's actually implicit: there's an extra "global NAME" or
"nonlocal NAME" in the equivalent code for a comprehension that isn't
there in the as-written source code, and doesn't get emitted for a
regular assignment expression or for the iteration variable in a
comprehension - it only shows up due to the defined interaction
between comprehensions and assignment expressions.

> One might as well complain
> that "import this" implicitly creates a local variable "this". Well,
> yes, it does, in a very loose sense, but that's what imports are
> defined as do and it is the whole purpose for making them.

And they behave the same way in every context where they're permitted to appear.

> If PEP 572's proposal goes ahead, the behaviour of assignment
> expressions will be *defined* as creating assignments in the local scope
> rather than the sublocal comprehension scope. To call that "implicit"
> is rather like saying that regular assignment is implicit.

I do say that regular assignments implicitly declare a name as local.
"Python has implicit local variable declarations" is also regularly
cited as one of the differences between it and languages that require
explicit declarations, like C. Even augmented assignments implicitly
declare a name as being a local (hence the infamous UnboundLocalError
that arises when you attempt to use an augmented assignment to rebind
a name from an outer scope).

The problem I have with PEP 572 is that it proposes *breaking that
otherwise universal pattern* - instead of having assignment
expressions in comprehensions implicitly declare the name as local in
the nested comprehension scope, it instead has them:

1. implicitly declare the name as global or as nonlocal in the
comprehension (or else raise an exception), depending on the nature of
the parent scope where the comprehension is used
2. in the nonlocal reference case, amend the symbol table analysis to
act like there was a preceding "if 0:\n for NAME in ():\n pass" in the
parent scope (so the compiler knows which outer function scope to
target)

The rationale being given for why that is OK is:

1. "Everyone" thinks comprehensions are just a for loop (even though
that hasn't been true for the better part of a decade, and was never
true for generator expressions)
2. If comprehensions are just a for loop, then assignment expressions
inside them should be local to the containing scope
3. Therefore the implicit declarations required to tie everything
together and allow folks to continue with an incorrect understanding
of how comprehensions work aren't really implicit - they're explicit
in the inaccurate expansion of the construct!

Can you imagine the reaction if anyone other than Guido or Tim was
attempting to argue for a change to the language that only makes sense
if we grant a completely inaccurate understanding of how a particular
language construct works as being a credible starting point?

Because that's how this comprehension scoping argument feels to me:

Proposal author: "If the language worked in a way other than it does,
then this proposal would make perfect sense."
Proposal reviewer: "Yes, but it doesn't work that way, it works this
way. We deliberately changed it because the old way caused problems."
Proposal author: "Ah, but it *used* to work that way, and a lot of
people still think it works that way, and we can get the compiler to
jump through hoops to pretend it still works that way, except for the
parts of the new way that we want to keep."
Proposal reviewer: "..."

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia