[Python-Dev] PEP 572: Assignment Expressions

[Chris Angelico]

On Wed, Apr 18, 2018 at 11:20 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Wed, Apr 18, 2018 at 10:13:58AM +1000, Chris Angelico wrote:
>
> [regarding comprehensions]
>
>> The changes here are only to edge and corner cases, other than as they
>> specifically relate to assignment expressions. The current behaviour
>> is intended to "do the right thing" according to people's
>> expectations, and it largely does so; those cases are not changing.
>> For list comprehensions at global or function scope, the ONLY case
>> that can change (to my knowledge) is where you reuse a variable name:
>>
>> [t for t in t.__parameters__ if t not in tvars]
>>
>> This works in 3.7 but will fail easily and noisily (UnboundLocalError)
>> with PEP 572.
>
> That's a major semantic change, and the code you show is no better or
> worse than:
>
>     t = ...
>     result = []
>     for t in t.parameters:
>         if t not in tvars:
>             result.append(t)
>
>
> which is allowed. I think you need a better justification for breaking
> it than merely the opinion:
>
>> IMO this is a poor way to write a loop,

Ah but that isn't what the list comp is equivalent to. If you want to
claim that "for t in t.parameters" is legal, you first have to assume
that you're overwriting t, not shadowing it. In the list comp as it is
today, the "for t in" part is inside an implicit nested function, but
the "t.parameters" part is outside that function.

Try this instead:

t = ...
def listcomp():
    result = []
    for t in t.parameters:
        if t not in tvars:
            result.append(t)
    return result
listcomp()

Except that it isn't that either, because the scope isn't quite that
clean. It actually involves a function parameter, and the iterator is
fetched before it's passed as a parameter, and then NOT fetched inside
the loop. So you actually can't write perfectly equivalent longhand.

PEP 572 will *reduce* the edge cases and complexity.

>> Note that the second of the open questions would actually return this
>> to current behaviour, by importing the name 't' into the local scope.
>
> Indeed. Maybe this needs to stop being an open question and become a
> settled question.

Okay. The question is open if you wish to answer it. Are you happy
with the extra complexity that this would entail? Discuss.

>> The biggest semantic change is to the way names are looked up at class
>> scope. Currently, the behaviour is somewhat bizarre unless you think
>> in terms of unrolling a loop *as a function*; there is no way to
>> reference names from the current scope, and you will instead ignore
>> the surrounding class and "reach out" into the next scope outwards
>> (probably global scope).
>>
>> Out of all the code in the stdlib, the *only* one that needed changing
>> was in Lib/typing.py, where the above comprehension was found. (Not
>> counting a couple of unit tests whose specific job is to verify this
>> behaviour.)
>
> If there are tests which intentionally verify this behaviour, that
> really hurts your position that the behaviour is an accident of
> implementation. It sounds like the behaviour is intended and required.

These changes also broke some tests of disassembly, which quote the
exact bytecode created for specific pieces of code. Does that mean
that we can't change anything? They're specifically verifying
(asserting) the behaviour that currently exists.

I've never tried to claim that this is going to have no impact. Of
course it will change things. Otherwise why have a PEP?

>> The potential for breakage is extremely low. Non-zero, but
>> far lower than the cost of introducing a new keyword, for instance,
>> which is done without deprecation cycles.
>
> Which new keywords are you thinking of? The most recent new keywords I
> can think of were "True/False", "as" and "with".

async, await? They became "soft keywords" and then full keywords, but
that's not exactly a deprecation period.

rosuav at sikorsky:~$ python3.5 -c "await = 1; print(await)"
1
rosuav at sikorsky:~$ python3.6 -c "await = 1; print(await)"
1
rosuav at sikorsky:~$ python3.7 -c "await = 1; print(await)"
  File "<string>", line 1
    await = 1; print(await)
          ^
SyntaxError: invalid syntax

The shift from 3.6 to 3.7 breaks any code that uses 'async' or 'await'
as an identifier. And that kind of breakage CAN happen in a minor
release. Otherwise, it would be virtually impossible to improve
anything in the language.

PEP 572 will make changes. The result of these changes will be fewer
complex or unintuitive interactions between different features in
Python, most notably comprehensions/genexps and class scope. It also
makes the transformation from list comp to external function more
accurate and easier to understand. For normal usage, the net result
will be the same, but the differences are visible if you actually
probe for them.

ChrisA