[Python-Dev] PEP 572 semantics

[Steve Dower]

Now that it's a done deal, I am closely reviewing the semantics section
of PEP 572. (I had expected one more posting of the final PEP, but it
seems the acceptance came somewhere in a thread that was already muted.)

Since there has been no final posting that I'm aware of, I'm referring
to https://www.python.org/dev/peps/pep-0572/ as of about an hour before
posting this (hopefully it doesn't take me that long).

To be clear, I am *only* looking at the "Syntax and semantics" section.
So if something has been written down elsewhere in the PEP, please take
my questions as a request to have it referenced from this section. I
also gave up on the discussion by the third python-dev thread - if there
were things decided that you think I'm stupid for not knowing, it
probably means they never made it into the PEP.

= Syntax and Semantics

Could we include the changes necessary to
https://docs.python.org/3/reference/grammar.html in order to specify
where these expressions are valid? And ideally check that they work.
This may expose new exceptional cases, but will also clarify some of the
existing ones, especially for those of us who write Python parsers.

== Exceptional cases

Are the cases in the "Exceptional cases" section supposed to raise
SyntaxError on compilation? That seems obvious, but no harm in stating
it. (FWIW, I'd vote to ban the "bad" cases in style guides or by forcing
parentheses, rather than syntactically. And for anyone who wonders why
that's different from my position on slashes in f-strings, it's because
I don't think we can ever resolve these cases but I hope that one day we
can fix f-string slashes :) )

== Scope of the target

The PEP uses the phrase "an assignment expression occurs in a
comprehension" - what does this mean? Does it occur when/where it is
compiled, instantiated, or executed? This is important because where it
occurs determines which scope will be modified. For sanity sake, I want
to assume that it means compiled, but now what happens when that scope
is gone?

>>> def f():
...     return (a := i for i in range(5))
...
>>> list(f())
[0, 1, 2, 3, 4]   # or a new error because the scope has gone?
>>> a
???

I'll push back real hard on doing the assignment in the scope where the
generator is executed:

>>> def do_secure_op(name, numbers):
...     authorised = check_authorised(name)
...     if not all(numbers):
...         raise ValueError()
...     if not authorised:
...         raise SecurityError()
...     print('You made it!')
...
>>> do_secure_op('whatever', (authorised := i for i in [1, 2, 3]))
You made it!
>>> authorised
NameError: name 'authorised' is undefined

>From the any()/all() examples, it seems clear that the target scope for
the assignment has to be referenced from the generator scope (but not
for other comprehension types, which can simply do one transfer of the
assigned name after fully evaluating all the contents). Will this
reference keep the frame object alive for as long as the generator
exists? Can it be a weak reference? Are assignments just going to be
silently ignored when the frame they should assign to is gone? I'd like
to see these clarified in the main text.


When an assignment is "expressly invalid" due to avoiding "edge cases",
does this mean we should raise a SyntaxError? Or a runtime error? I'm
not sure how easily these can be detected by our current compiler (or
runtime, for that matter), but in the other tools that I work on it
isn't going to be a trivial check.

Also, I'm not clear at all on why [i := i+1 for i in range(5)] is a
problem? Similarly for the other examples here. There's nothing wrong
with `for i in range(5): i = i+1`, so why forbid this?

== Relative precedence

"may be used directly in a positional function call argument" - why not
use the same syntax as generator expressions? Require parentheses unless
it's the only argument. It seems like that's still got a TODO on it from
one of the examples, so consider this a vote for matching
generator-as-argument syntax.


== Differences between assignment expressions

I'm pretty sure the equivalent of "x = y = z = 0" would be "z := (y :=
(x := 0))". Not that it matters when there are no side-effects of
assignment (unless we decide to raise at runtime for invalid
assignments), but it could become a point of confusion for people in the
future to see it listed like this. Assignment expressions always
evaluate from innermost to outermost.

Gramatically, "Single assignment targets *other than* NAME are not
supported" would be more precise. And for specification's sake, does
"not supported" mean "is a syntax error"?

The "equivalent needs extra parentheses" examples add two sets of extra
parentheses. Are both required? Or just the innermost set?

---

Apologies for the lack of context. I've gone back and added the section
headings for as I read through this section.

Cheers,
Steve