What's up with assignment expression and tuples?

Hello, Everyone knows how hard to find a compelling usecase for the assignment expression operator (":=", colloquially "walrus operator"). https://www.python.org/dev/peps/pep-0572/ examples never felt compelling and we all remember the split it caused. I finally found a usecase where *not* using assignment expression is *much* worse than using it. I'm working on SSA (Static Single Assignment, https://en.wikipedia.org/wiki/Static_single_assignment_form) conversion of Python programs, where there's a need to "join" dataflow of values from different control flow paths using a special function (Phi function). This "joining" itself creates a new variable, and of course, the original variable was used in an expression. We've got assignment in expression, assignment expression operator to the rescue! With it, a simple loop like: ---- a = 0 while a < 5: a += 1 ---- becomes: ---- a0 = 0 while (a1 := phi(a0, a2)) < 5: a2 = a1 + 1 ---- So far, so good. But semantics of Phi function is parallel assignment. No problem with Python either, "a, b = b, c" is exactly parallel assignment. So, let's try example with 2 variables: ---- a = 0 b = 10 while a < 5: a += 1 b += 1 ---- becomes: ---- a0 = 0 b0 = 10 while ((a1, b1) := phi([a0, a2], [b0, b2]))[0] < 5: a2 = a1 + 1 b2 = b1 + 1 ---- But oops:
SyntaxError: cannot use assignment expressions with tuple
To reproduce in the REPL: ----
Why this accidental syntactic gap? Why assignment statement can do parallel assignment with a tuple on LHS, and assignment operator suddenly can't? Why the adhoc naming and conceptual shift on the AST level, when PEP572 explicitly talks about *assignment operator*, but corresponding node on the AST level is called NamedExpr? Why look at assignment expression as "name of expression" instead of assignment expression per se? It's of course not a problem to recast: NamedExpr(expr target, expr value) to NamedExpr(expr* target, expr value) in the ASDL (and it works out of the box), the point is that it should have been ExprAssign from the start (folloing the AugAssign and AnnAssign tradition). -- Best regards, Paul mailto:pmiscml@gmail.com

On 2/5/2021 2:51 AM, Paul Sokolovsky wrote:
As should be clear from reading "Differences between assignment expressions and assignment statements", this 'gap' in entirely intentional, not accidental. *All* elaborations of 'name := expression' are listed and rejected as outside the scope of the proposal, which was to keep one reference to the expression value for later use. At least some of these elaborations were suggested and rejected during the voluminous discussion. The principal a.e. use in conditional expressions is testing for non-nullness. Your
is an unusual and very specific use. You want to have your tuple (for subscripting for testing) and eat it too (by unpacking). One can instead separate unpacking from testing a couple of ways while (tup := phi([a0, a2], [b0, b2]))[0] < 5: a2, b2 = tup a2 = a1 + 1 b2 = b1 + 1 while True: a1, b1 = phi([a0, a2], [b0, b2]) if a1 >= 5: break a2 = a1 + 1 b2 = b1 + 1 -- Terry Jan Reedy

Hello, Thanks for the reply. On Fri, 5 Feb 2021 13:32:25 -0500 Terry Reedy <tjreedy@udel.edu> wrote:
And looking back now, that seems like intentionally added accidental gap in the language (https://en.wikipedia.org/wiki/Accidental_gap). Similar to artificially limiting decorator syntax, which was already un-limited. But seems, there're no "lessons learned", and there's now need to wait a decade again before fixing that?
Well, many people were thinking (and I bet still think) that ":=" itself is very unusual case. But if it's in, why not make it consistent with the assignment statement and unleash the full power of it?
You want to have your tuple (for subscripting for testing) and eat it too (by unpacking).
That's good characterization, thanks. And Python syntax alone would allow to do that, if not extra-syntactical limitations put on top of it.
Right, but I started my original email with "I finally found a usecase where *not* using assignment expression is *much* worse than using it." Both conversions above apply additional disturbances to the original program, beyond pure SSA conversion, which is quite a disturbance on its own. I was really excited that Python 3.7+ would be *the* language which would allow to express SSA conversion faithfully on the source form of the high-level language (usually SSA is applied to low-level assembly-like intermediate representation). But oops, accidental gap, and despite all the Python advanceness, still need to apply workarounds as with other mundane languages. [] -- Best regards, Paul mailto:pmiscml@gmail.com

On 5/02/21 8:51 pm, Paul Sokolovsky wrote:
SSA seems to be something intended for compilers to use as an intermediate representation, not something to actually write code in. So I'm puzzled as to why you would want to do this. Also, what definition do you have in mind for phi? It doesn't seem to be something you can implement as a real function. -- Greg

Hello, On Sat, 06 Feb 2021 12:35:09 +1300 Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Let's summarize what I've written so far: 1. I draw attention to the case that "=" can do parallel assignments, while ":=". If I would stop here, there would be immediate questions: "But where/how you would use that?". So I at once presented: 2. A usecase, and even not a manual once-off usecase, but a usecase which would affect every "while", again, and again. Now you definitely can question *the usecase*, but please keep in mind that my point was to show a gap in walrus functionality, and just show an example for that. But back to the specific SSA usecase: 1. Before compilers can "use it as an intermediate representation", humans should code up those compilers. For that, they would need to learn how SSA looks and play with it. And I for example was very excited at the perspective that Python is *the* language which would allow to express pure SSA form on the high-level language level (without additional patchings, not related to SSA per se). 2. Even after compilers produce SSA as IR, humans still need to have insight into it and verify it. Again, ability to do *just SSA* and not "munge code and do SSA" is a benefit. Note that "but how that will work on a millions lines of generated code" is not infrequent question when discussing various Python proposals, so please don't dismiss the SSA case just because it's supposed to be "auto-generated IR". It's still need to be legible and debuggable for humans.
Also, what definition do you have in mind for phi? It doesn't seem to be something you can implement as a real function.
With the above notes, I guess discussing the SSA encoding case in more detail specifically on python-dev doesn't make much sense. But you're absolutely right, I simplified snippets for the presentation here. Real "executable SSA" code requires more arguments to phi, and is available in this repo: https://github.com/pfalcon/python-ssa/blob/master/example_while1_ssa.py (As you can see, I exactly was doing "conversion to the infinite loop" transformation, as also suggested by Terry Reedy. But I'd like to use the SSA usecase to try to address accidental gap in the walrus operator functionality, that's the point).
-- Greg
[] -- Best regards, Paul mailto:pmiscml@gmail.com

05.02.21 09:51, Paul Sokolovsky пише:
Such code quickly becomes unreadable. Especially if in real code function has additional arguments and names are longer that 2-3 characters. The following code is not much larger but more clear and extensible: a0 = 0 b0 = 10 while True: a1, b1 := phi([a0, a2], [b0, b2])) if b1 >= 5: break a2 = a1 + 1 b2 = b1 + 1

Hello, On Sat, 6 Feb 2021 10:46:54 +0200 Serhiy Storchaka <storchaka@gmail.com> wrote:
But at least it fully corresponds to the original statements and control flow of the program, that's the point!
As I mentioned in other replies, that's what I've been doing. But that does *NOT* correspond to the original program, or SSA conversion of it. Instead, it's SSA + random munging. And when debugging SSA, you may argue what's worse: to look at 3-stories phi's, or to look at the code structure which doesn't correspond to the input code. But that's not the point. I now in https://bugs.python.org/issue43143 gave more down to earth example: For as long as you agree that following is acceptable: func(a := val) , then I'd say it really doesn't make much sense to argue that following should not be acceptable: min((b, a) := (a, b)) -- Best regards, Paul mailto:pmiscml@gmail.com

Hi Paul, I suggest that you just go straight to the PEP phase. --Guido On Thu, Feb 4, 2021 at 11:54 PM Paul Sokolovsky <pmiscml@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>

Hello, On Sun, 7 Feb 2021 13:10:55 -0800 Guido van Rossum <guido@python.org> wrote:
Hi Paul,
I suggest that you just go straight to the PEP phase.
Thanks. In all fairness, I don't expect immediate resolution to this issue. But I'm aware of out for at least a year, and keep returning to it (yes, in context of my SSA experiments). So, I proceeded to the next stage - bring it up on its own, then created a bug ticket: https://bugs.python.org/issue43143 And my idea is to try to argue that it would be just a "grammar bugfix", similarly to already existing grammar elaborations for walrus: https://bugs.python.org/issue42316 https://bugs.python.org/issue42374 https://bugs.python.org/issue42381 Granted, allowing "foo((a, b) := (b, a))" is a bit bigger change than allowing "foo[a := b]" instead of "foo[(a := b)]". But if people find assigning within index expression useful, up to reporting it, and then other people ack it and fix it, then why not fix parallel assignment case? Implementation-wise they *seem* to be of the similar effort/complexity - just a one-term grammar change. (I still need to run the testsuite, yeah).
-- Best regards, Paul mailto:pmiscml@gmail.com

On 2/5/2021 2:51 AM, Paul Sokolovsky wrote:
As should be clear from reading "Differences between assignment expressions and assignment statements", this 'gap' in entirely intentional, not accidental. *All* elaborations of 'name := expression' are listed and rejected as outside the scope of the proposal, which was to keep one reference to the expression value for later use. At least some of these elaborations were suggested and rejected during the voluminous discussion. The principal a.e. use in conditional expressions is testing for non-nullness. Your
is an unusual and very specific use. You want to have your tuple (for subscripting for testing) and eat it too (by unpacking). One can instead separate unpacking from testing a couple of ways while (tup := phi([a0, a2], [b0, b2]))[0] < 5: a2, b2 = tup a2 = a1 + 1 b2 = b1 + 1 while True: a1, b1 = phi([a0, a2], [b0, b2]) if a1 >= 5: break a2 = a1 + 1 b2 = b1 + 1 -- Terry Jan Reedy

Hello, Thanks for the reply. On Fri, 5 Feb 2021 13:32:25 -0500 Terry Reedy <tjreedy@udel.edu> wrote:
And looking back now, that seems like intentionally added accidental gap in the language (https://en.wikipedia.org/wiki/Accidental_gap). Similar to artificially limiting decorator syntax, which was already un-limited. But seems, there're no "lessons learned", and there's now need to wait a decade again before fixing that?
Well, many people were thinking (and I bet still think) that ":=" itself is very unusual case. But if it's in, why not make it consistent with the assignment statement and unleash the full power of it?
You want to have your tuple (for subscripting for testing) and eat it too (by unpacking).
That's good characterization, thanks. And Python syntax alone would allow to do that, if not extra-syntactical limitations put on top of it.
Right, but I started my original email with "I finally found a usecase where *not* using assignment expression is *much* worse than using it." Both conversions above apply additional disturbances to the original program, beyond pure SSA conversion, which is quite a disturbance on its own. I was really excited that Python 3.7+ would be *the* language which would allow to express SSA conversion faithfully on the source form of the high-level language (usually SSA is applied to low-level assembly-like intermediate representation). But oops, accidental gap, and despite all the Python advanceness, still need to apply workarounds as with other mundane languages. [] -- Best regards, Paul mailto:pmiscml@gmail.com

On 5/02/21 8:51 pm, Paul Sokolovsky wrote:
SSA seems to be something intended for compilers to use as an intermediate representation, not something to actually write code in. So I'm puzzled as to why you would want to do this. Also, what definition do you have in mind for phi? It doesn't seem to be something you can implement as a real function. -- Greg

Hello, On Sat, 06 Feb 2021 12:35:09 +1300 Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Let's summarize what I've written so far: 1. I draw attention to the case that "=" can do parallel assignments, while ":=". If I would stop here, there would be immediate questions: "But where/how you would use that?". So I at once presented: 2. A usecase, and even not a manual once-off usecase, but a usecase which would affect every "while", again, and again. Now you definitely can question *the usecase*, but please keep in mind that my point was to show a gap in walrus functionality, and just show an example for that. But back to the specific SSA usecase: 1. Before compilers can "use it as an intermediate representation", humans should code up those compilers. For that, they would need to learn how SSA looks and play with it. And I for example was very excited at the perspective that Python is *the* language which would allow to express pure SSA form on the high-level language level (without additional patchings, not related to SSA per se). 2. Even after compilers produce SSA as IR, humans still need to have insight into it and verify it. Again, ability to do *just SSA* and not "munge code and do SSA" is a benefit. Note that "but how that will work on a millions lines of generated code" is not infrequent question when discussing various Python proposals, so please don't dismiss the SSA case just because it's supposed to be "auto-generated IR". It's still need to be legible and debuggable for humans.
Also, what definition do you have in mind for phi? It doesn't seem to be something you can implement as a real function.
With the above notes, I guess discussing the SSA encoding case in more detail specifically on python-dev doesn't make much sense. But you're absolutely right, I simplified snippets for the presentation here. Real "executable SSA" code requires more arguments to phi, and is available in this repo: https://github.com/pfalcon/python-ssa/blob/master/example_while1_ssa.py (As you can see, I exactly was doing "conversion to the infinite loop" transformation, as also suggested by Terry Reedy. But I'd like to use the SSA usecase to try to address accidental gap in the walrus operator functionality, that's the point).
-- Greg
[] -- Best regards, Paul mailto:pmiscml@gmail.com

05.02.21 09:51, Paul Sokolovsky пише:
Such code quickly becomes unreadable. Especially if in real code function has additional arguments and names are longer that 2-3 characters. The following code is not much larger but more clear and extensible: a0 = 0 b0 = 10 while True: a1, b1 := phi([a0, a2], [b0, b2])) if b1 >= 5: break a2 = a1 + 1 b2 = b1 + 1

Hello, On Sat, 6 Feb 2021 10:46:54 +0200 Serhiy Storchaka <storchaka@gmail.com> wrote:
But at least it fully corresponds to the original statements and control flow of the program, that's the point!
As I mentioned in other replies, that's what I've been doing. But that does *NOT* correspond to the original program, or SSA conversion of it. Instead, it's SSA + random munging. And when debugging SSA, you may argue what's worse: to look at 3-stories phi's, or to look at the code structure which doesn't correspond to the input code. But that's not the point. I now in https://bugs.python.org/issue43143 gave more down to earth example: For as long as you agree that following is acceptable: func(a := val) , then I'd say it really doesn't make much sense to argue that following should not be acceptable: min((b, a) := (a, b)) -- Best regards, Paul mailto:pmiscml@gmail.com

Hi Paul, I suggest that you just go straight to the PEP phase. --Guido On Thu, Feb 4, 2021 at 11:54 PM Paul Sokolovsky <pmiscml@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>

Hello, On Sun, 7 Feb 2021 13:10:55 -0800 Guido van Rossum <guido@python.org> wrote:
Hi Paul,
I suggest that you just go straight to the PEP phase.
Thanks. In all fairness, I don't expect immediate resolution to this issue. But I'm aware of out for at least a year, and keep returning to it (yes, in context of my SSA experiments). So, I proceeded to the next stage - bring it up on its own, then created a bug ticket: https://bugs.python.org/issue43143 And my idea is to try to argue that it would be just a "grammar bugfix", similarly to already existing grammar elaborations for walrus: https://bugs.python.org/issue42316 https://bugs.python.org/issue42374 https://bugs.python.org/issue42381 Granted, allowing "foo((a, b) := (b, a))" is a bit bigger change than allowing "foo[a := b]" instead of "foo[(a := b)]". But if people find assigning within index expression useful, up to reporting it, and then other people ack it and fix it, then why not fix parallel assignment case? Implementation-wise they *seem* to be of the similar effort/complexity - just a one-term grammar change. (I still need to run the testsuite, yeah).
-- Best regards, Paul mailto:pmiscml@gmail.com
participants (5)
-
Greg Ewing
-
Guido van Rossum
-
Paul Sokolovsky
-
Serhiy Storchaka
-
Terry Reedy