[Python-Dev] PEP 572: Assignment Expressions

Tue Apr 17 10:01:50 EDT 2018

On Tue, Apr 17, 2018 at 10:17 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Initially I thought the problem was specific to tuple unpacking
> syntax, but attempting to explain why subscript assignment and
> attribute assignments were OK made me realise that they're actually
> even worse off (since they can execute arbitrary code on both setting
> and retrieval, whereas tuple unpacking only iterates over iterables).
>
> Tackling those in order...
>
> Tuple unpacking:
>
>     What's the result type for "a, b, c := range(3)"? Is it a range()
> object? Or is it a 3-tuple? If it's a 3-tuple, is that 3-tuple "(1, 2,
> 3)" or "(a, b, range(3))"?
>     Once you have your answer, what about "a, b, c := iter(range(3))"
> or "a, b, *c := range(10)"?

This is one that I didn't originally think about, and when I first
tried it out a couple of weeks ago, I decided against mentioning it
either way, because I've no idea what _should_ be done. But here's
what my reference implementation does:

>>> x = (a,b,c := range(3))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'a' is not defined

In other words, it's being parsed as:

x = (a, b, (c := range(3)))

Forcing the interpretation does work:

>>> x = ((a,b,c) := range(3))

And then the value of the expression is the same object that just got
assigned (or in this case, unpacked):

>>> x
range(0, 3)

That's true even if it's an exhausted iterator:

>>> x = ((a,b,c) := iter(range(3)))
>>> x
<range_iterator object at 0x7fb8e52c7030>
>>> next(x)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

The way it works is that the RHS gets evaluated, then that gets put on
ice for a moment, and the assignment done with a copy of it. (In the
CPython reference implementation, "put on ice" is done with DUP_TOP.)
So the value that got assigned - even if that assignment involved
unpacking a sequence - is still passed through the assignment and out
the other side.

>>> dis.dis("x = ((a,b,c) := range(3))")
### evaluate RHS
  1           0 LOAD_NAME                0 (range)
              2 LOAD_CONST               0 (3)
              4 CALL_FUNCTION            1
### grab another reference to the range
              6 DUP_TOP
### do the assignment
              8 UNPACK_SEQUENCE          3
             10 STORE_NAME               1 (a)
             12 STORE_NAME               2 (b)
             14 STORE_NAME               3 (c)
### carry on with the rest of the expression
             16 STORE_NAME               4 (x)
             18 LOAD_CONST               1 (None)
             20 RETURN_VALUE

> Whichever answers we chose would be surprising at least some of the
> time, so it seems simplest to disallow such ambiguous constructs, such
> that the only possible interpretation is as "(a, b, range(3))"

Yeah, that's what happens. Tuple display is defined as "test , test"
and a 'test' can be 'target := value', so each element of a tuple can
have an assignment expression in it. If you actually want to unpack
inside an assignment expression, you need parentheses.

> Subscript assignment:
>
>     What's the final value of "result" in "seq = list(); result =
> (seq[:] := range(3))"? Is it "range(3)"? Or is it "[1, 2, 3]"?
>     As for tuple unpacking, does your preferred answer change for the
> case of "seq[:] := iter(range(3))"?

It's range(3), and no, my preferred answer doesn't change. It'll
probably never be useful to unpack an iterator in an assignment
expression (since you'll always get an exhausted iterator at the end
of it), but I'm sure there'll be uses for unpacking iterables.

>     More generally, if I write  "container[k] := value", does only
> "type(container).__setitem__" get called, or does
> "type(container).__getitem__" get called as well?

Only setitem. If you like, imagine the := operator as a tee junction:
you "tap off" the pipe and snag it with assignment, and also keep
using it just as if the assignment hadn't been there.

> Again, this seems inherently ambiguous to me, and hence best avoided
> (at least for now), such that the result is always unambiguously
> "range(3)".
>
> Attribute assignment:
>
>     If I write  "obj.attr := value", does only "type(obj).__setattr__"
> get called, or does "type(obj).__getattribute__" get called as well?

I didn't change anything about how assignment actually works, so I
would expect it to be exactly the same semantics as
statement-assignment has. Let's test.

>>> class X:
...   @property
...   def spam(self): return 42
...   @spam.setter
...   def spam(self, val): print("Setting spam to", val)
...
>>> x = X()
>>> dis.dis("(x.spam := 7)")
  1           0 LOAD_CONST               0 (7)
              2 DUP_TOP
              4 LOAD_NAME                0 (x)
              6 STORE_ATTR               1 (spam)
              8 RETURN_VALUE
>>> (x.spam := 7)
Setting spam to 7
7

Looks good to me. If I had to choose semantics, I don't think this
would be a bad choice; and for something that derived naturally from a
basic "let's just copy in the code for assignment", it's looking
consistent and usable.

> While I can't think of a simple obviously ambiguous example using
> builtins or the standard library, result ambiguity exists even for the
> attribute access case, since type or value coercion may occur either
> when setting the attribute, or when retrieving it, so it makes a
> difference as to whether a reference to the right hand side is passed
> through directly as the assignment expression result, or if the
> attribute is stored and then retrieved again.

Agreed.

> That ambiguity generally doesn't exist with simple name bindings (I'm
> excluding execution namespaces with exotic binding behaviour from
> consideration here, as the consequences of trying to work with those
> are clearly on the folks defining and using them).

The cool thing about the simple and naive code is that even those
should work. I don't have an example ready for demo, but I fully
expect that it would 'just work' the exact same way; the namespace
would never be retrieved from, only set to.

Hmm. I don't know what the consequences would be on class namespace
with a non-vanilla dict. Probably functionally identical. But there
might be some extremely weird cases if the namespace dict accepts
setitem and then raises KeyError for that key.

>> Style guide recommendations
>> ===========================
>>
>> As this adds another way to spell some of the same effects as can already be
>> done, it is worth noting a few broad recommendations. These could be included
>> in PEP 8 and/or other style guides.
>>
>> 1. If either assignment statements or assignment expressions can be
>>    used, prefer statements; they are a clear declaration of intent.
>>
>> 2. If using assignment expressions would lead to ambiguity about
>>    execution order, restructure it to use statements instead.
>>
>> 3. Chaining multiple assignment expressions should generally be avoided.
>>    More than one assignment per expression can detract from readability.
>
> Given the many different uses for ":" identified on python-ideas, I'm
> inclined to suggest making these proposed style guidelines more
> prescriptive (at least initially) by either:
>
> 1. Listing out specific approved unambiguous use cases (i.e. if
> statement conditions, while loop conditions, list comprehensions,
> generation expressions)
> 2. Making the 3rd admonition more general by advising against using
> ":" for more than one purpose in the same expression (i.e. don't
> combine assignment expressions with slicing syntax, lambda
> expressions, function headers, variable annotations, dict or set
> displays, dict or set comprehensions)

I'm actually dubious about the third point as it stands. It's either
too broad or too narrow, but I'm not sure which; there are plenty of
legitimate uses for multiple colons in an expression without
confusion, but there are also plenty of ways that even a single
assignexp could be pretty bad for readability. So I'm hoping that we
can get some people to test this out well before 3.8 lands, and refine
the style recommendations before this feature hits release.

ChrisA