[Python-ideas] PEP 572: Assignment Expressions (post #4)

[Clint Hepner]

> On 2018 Apr 11 , at 1:32 a, Chris Angelico <rosuav at gmail.com> wrote:
> 
> Wholesale changes since the previous version. Statement-local name
> bindings have been dropped (I'm still keeping the idea in the back of
> my head; this PEP wasn't the first time I'd raised the concept), and
> we're now focusing primarily on assignment expressions, but also with
> consequent changes to comprehensions.

Overall, I'm slightly negative on this. I think named expressions will
be a good thing to have, but not in this form. I'll say up front that,
being fully aware of the issues surrounding the introduction of a new
keyword, something like a let expression in Haskell would be more readable
than embedded assignments in most cases.

In the end, I suspect my `let` proposal is a nonstarter and just useful
to list with the rest of the rejected alternatives, but I wanted.

   
> 
> Abstract
> ========
> 

[...]

> 
> 
> Rationale
> =========
> 

[...]

> 
> Syntax and semantics
> ====================
> 
> In any context where arbitrary Python expressions can be used, a **named
> expression** can appear. This can be parenthesized for clarity, and is of
> the form ``(target := expr)`` where ``expr`` is any valid Python expression,
> and ``target`` is any valid assignment target.
> 
> The value of such a named expression is the same as the incorporated
> expression, with the additional side-effect that the target is assigned
> that value.
> 
>    # Similar to the boolean 'or' but checking for None specifically
>    x = "default" if (eggs := spam().ham) is None else eggs

> 
>    # Even complex expressions can be built up piece by piece
>    y = ((eggs := spam()), (cheese := eggs.method()), cheese[eggs])

I find the assignments make it difficult to pick out what the final expression looks like.
The first isn't too bad, but it took me a moment to figure out what y was. Quick: is it

  * (a, b, c)
  * (a, (b, c))
  * ((a, b), c)
  * something else

First I though it was (a, b, c), then I thought it was actually ((a, b), c), before
carefully counting the parentheses showed that I was right the first time.

These would be clearer if you could remove the assignment from the expression itself.
Assuming "let" were available as a keyword,

    x = (let eggs = spam().ham
         in
         "default" if eggs is None else eggs)
    y = (let eggs = spam(),
             cheese = eggs.method()
         in
         (eggs, cheese, cheese[eggs]))

Allowing for differences in how best to format such an expression, the final
expression is clearly separate from its component assignment. (More on this
in the Alternative Spellings section below.)

> 
> Differences from regular assignment statements
> ----------------------------------------------
> 
> An assignment statement can assign to multiple targets::
> 
>    x = y = z = 0
> 
> To do the same with assignment expressions, they must be parenthesized::
> 
>    assert 0 == (x := (y := (z := 0)))

There's no rationale given for why this must be parenthesized. 
If := were right-associative,

    assert 0 == (x := y := z := 0)

would work fine. (With high enough precedence, the remaining parentheses
could be dropped, but one would probably keep them for clarity.)
I think you need to spell out its associativity and precedence in more detail,
and explain why the rationale for the choice made.

> 
> Augmented assignment is not supported in expression form::
> 
>>>> x +:= 1
>      File "<stdin>", line 1
>        x +:= 1
>            ^
>    SyntaxError: invalid syntax

There's no reason give for why this is invalid. I assume it's a combination
of 1) Having both += and +:=/:+= would be redundant and 2) not wanting
to add 11+ new operators to the language.

> 
> Otherwise, the semantics of assignment are unchanged by this proposal.
> 

[List comprehensions deleted]

> 
> 
> Recommended use-cases
> =====================
> 
> Simplifying list comprehensions
> -------------------------------
> 
> These list comprehensions are all approximately equivalent::

[existing alternatives redacted]

>    # Using a temporary name
>    stuff = [[y := f(x), x/y] for x in range(5)]

Again, this would be clearer if the assignment were separated from the expression where it
would be used.

    stuff = [let y = f(x) in [y, x/y] for x in range(5)]


> 
> Capturing condition values
> --------------------------
> 
> Assignment expressions can be used to good effect in the header of
> an ``if`` or ``while`` statement::
> 
>    # Current Python, not caring about function return value
>    while input("> ") != "quit":
>        print("You entered a command.")
> 
>    # Current Python, capturing return value - four-line loop header
>    while True:
>        command = input("> ");
>        if command == "quit":
>            break
>        print("You entered:", command)
> 
>    # Proposed alternative to the above
>    while (command := input("> ")) != "quit":
>        print("You entered:", command)
> 
>    # Capturing regular expression match objects
>    # See, for instance, Lib/pydoc.py, which uses a multiline spelling
>    # of this effect
>    if match := re.search(pat, text):
>        print("Found:", match.group(0))
> 
>    # Reading socket data until an empty string is returned
>    while data := sock.read():
>        print("Received data:", data)
> 
> Particularly with the ``while`` loop, this can remove the need to have an
> infinite loop, an assignment, and a condition. It also creates a smooth
> parallel between a loop which simply uses a function call as its condition,
> and one which uses that as its condition but also uses the actual value.

These are the most compelling examples so far, doing the most to push me
towards a +1. In particular, my `let` expression is too verbose here:

    while let data = sock.read() in data:
        print("Received data:", data)

I have an idea in the back of my head about `NAME := FOO` being syntactic
sugar for `let NAME = FOO in FOO`, but it's not well thought out.
    

> 
> 
> Rejected alternative proposals
> ==============================
> 
> Proposals broadly similar to this one have come up frequently on python-ideas.
> Below are a number of alternative syntaxes, some of them specific to
> comprehensions, which have been rejected in favour of the one given above.
> 
> 
> Alternative spellings
> ---------------------
> 
> Broadly the same semantics as the current proposal, but spelled differently.
> 
> 1. ``EXPR as NAME``, with or without parentheses::
> 
>       stuff = [[f(x) as y, x/y] for x in range(5)]
> 
>   Omitting the parentheses in this form of the proposal introduces many
>   syntactic ambiguities.  Requiring them in all contexts leaves open the
>   option to make them optional in specific situations where the syntax is
>   unambiguous (cf generator expressions as sole parameters in function
>   calls), but there is no plausible way to make them optional everywhere.
> 
>   With the parentheses, this becomes a viable option, with its own tradeoffs
>   in syntactic ambiguity.  Since ``EXPR as NAME`` already has meaning in
>   ``except`` and ``with`` statements (with different semantics), this would
>   create unnecessary confusion or require special-casing.
> 
> 2. Adorning statement-local names with a leading dot::
> 
>       stuff = [[(f(x) as .y), x/.y] for x in range(5)] # with "as"
>       stuff = [[(.y := f(x)), x/.y] for x in range(5)] # with ":="
> 
>   This has the advantage that leaked usage can be readily detected, removing
>   some forms of syntactic ambiguity.  However, this would be the only place
>   in Python where a variable's scope is encoded into its name, making
>   refactoring harder.  This syntax is quite viable, and could be promoted to
>   become the current recommendation if its advantages are found to outweigh
>   its cost.
> 
> 3. Adding a ``where:`` to any statement to create local name bindings::
> 
>       value = x**2 + 2*x where:
>           x = spam(1, 4, 7, q)
> 
>   Execution order is inverted (the indented body is performed first, followed
>   by the "header").  This requires a new keyword, unless an existing keyword
>   is repurposed (most likely ``with:``).  See PEP 3150 for prior discussion
>   on this subject (with the proposed keyword being ``given:``).

4. Adding a ``let`` expression to create local bindings

    value = let x = spam(1, 4, 7, q) in x**2 + 2*x

5. Adding a ``where`` expression to create local bindings:

    value = x**2 + 2*x where x = spam(1, 4, 7, q)

Both have the extra-keyword problem. Multiple bindings are little harder
to add than they would be with the ``where:`` modifier, although
a few extra parentheses and judicious line breaks make it not so bad to
allow a comma-separated list, as shown in my first example at the top of
this reply.

> 
> 
> Special-casing conditional statements
> -------------------------------------
> 
> One of the most popular use-cases is ``if`` and ``while`` statements.  Instead
> of a more general solution, this proposal enhances the syntax of these two
> statements to add a means of capturing the compared value::
> 
>    if re.search(pat, text) as match:
>        print("Found:", match.group(0))
> 
> This works beautifully if and ONLY if the desired condition is based on the
> truthiness of the captured value.  It is thus effective for specific
> use-cases (regex matches, socket reads that return `''` when done), and
> completely useless in more complicated cases (eg where the condition is
> ``f(x) < 0`` and you want to capture the value of ``f(x)``).  It also has
> no benefit to list comprehensions.
> 
> Advantages: No syntactic ambiguities. Disadvantages: Answers only a fraction
> of possible use-cases, even in ``if``/``while`` statements.
> 
> 
> Special-casing comprehensions
> -----------------------------
> 
> Another common use-case is comprehensions (list/set/dict, and genexps). As
> above, proposals have been made for comprehension-specific solutions.
> 
> 1. ``where``, ``let``, or ``given``::
> 
>       stuff = [(y, x/y) where y = f(x) for x in range(5)]
>       stuff = [(y, x/y) let y = f(x) for x in range(5)]
>       stuff = [(y, x/y) given y = f(x) for x in range(5)]
> 
>   This brings the subexpression to a location in between the 'for' loop and
>   the expression. It introduces an additional language keyword, which creates
>   conflicts. Of the three, ``where`` reads the most cleanly, but also has the
>   greatest potential for conflict (eg SQLAlchemy and numpy have ``where``
>   methods, as does ``tkinter.dnd.Icon`` in the standard library).
> 
> 2. ``with NAME = EXPR``::
> 
>       stuff = [(y, x/y) with y = f(x) for x in range(5)]
> 
>   As above, but reusing the `with` keyword. Doesn't read too badly, and needs
>   no additional language keyword. Is restricted to comprehensions, though,
>   and cannot as easily be transformed into "longhand" for-loop syntax. Has
>   the C problem that an equals sign in an expression can now create a name
>   binding, rather than performing a comparison. Would raise the question of
>   why "with NAME = EXPR:" cannot be used as a statement on its own.
> 
> 3. ``with EXPR as NAME``::
> 
>       stuff = [(y, x/y) with f(x) as y for x in range(5)]
> 
>   As per option 2, but using ``as`` rather than an equals sign. Aligns
>   syntactically with other uses of ``as`` for name binding, but a simple
>   transformation to for-loop longhand would create drastically different
>   semantics; the meaning of ``with`` inside a comprehension would be
>   completely different from the meaning as a stand-alone statement, while
>   retaining identical syntax.
> 
> Regardless of the spelling chosen, this introduces a stark difference between
> comprehensions and the equivalent unrolled long-hand form of the loop.  It is
> no longer possible to unwrap the loop into statement form without reworking
> any name bindings.  The only keyword that can be repurposed to this task is
> ``with``, thus giving it sneakily different semantics in a comprehension than
> in a statement; alternatively, a new keyword is needed, with all the costs
> therein.

4. `` let NAME = EXPR1  in EXPR2``::

    stuff = [let y = f(x) in (y, x/y) for x in range(5)]

I don't have anything new to say about this. It has the same keyword
objections as similar proposals, and I think I've addressed the use case
elsewhere.
 
> 
> Frequently Raised Objections
> ============================
> 
> Why not just turn existing assignment into an expression?
> ---------------------------------------------------------
> 
> C and its derivatives define the ``=`` operator as an expression, rather than
> a statement as is Python's way.  This allows assignments in more contexts,
> including contexts where comparisons are more common.  The syntactic similarity
> between ``if (x == y)`` and ``if (x = y)`` belies their drastically different
> semantics.  Thus this proposal uses ``:=`` to clarify the distinction.
> 

> 
> With assignment expressions, why bother with assignment statements?
> -------------------------------------------------------------------
> 
> The two forms have different flexibilities.  The ``:=`` operator can be used
> inside a larger expression; the ``=`` operator can be chained more
> conveniently, and closely parallels the inline operations ``+=`` and friends.
> The assignment statement is a clear declaration of intent: this value is to
> be assigned to this target, and that's it.

I don't find this convincing. I don't really see chained assignments often enough
to worry about how they are written, plus note my earlier question about the
precedence and associativity of :=.

The fact is, `x := 5` as an expression statement appears equivalent to the
assignment statement `x = 5`, so I suspect people will start using it as such
no matter how strongly you suggest they shouldn't.

--
Clint