[Python-ideas] Spelling of Assignment Expressions PEP 572 (was post #4)

[Nick Coghlan]

On 13 April 2018 at 23:18, Steven D'Aprano <steve at pearwood.info> wrote:
> On Fri, Apr 13, 2018 at 09:56:35PM +1000, Chris Angelico wrote:
>
>
>> How many times have people asked for "with (expr as name):" to
>> be supported, allowing the statement to spread over multiple lines?
>> With this syntax, it would suddenly be permitted - with dangerously
>> similar semantics.
>
> I see your point, but why don't we support "with (expr as name):" to
> allow multiple lines? No, don't answer that... its off-topic. Forget I
> asked.

It's not completely off topic. as it's due to the fact we use "," to
separate both context managers and items in a tuple, so "with (cm1,
cm2, cm3):" is currently legal syntax that means something quite
different from "with cm1, cm2, cm3:". While using the parenthesised
form is *pointless* (since it will blow up at runtime due to tuples
not being context managers), the fact it's syntactically valid makes
us more hesitant to add the special case around parentheses handling
than we were for import statements. The relevance to PEP 572 is as a
reminder that since we *really* don't like to add yet more different
cases to "What do parentheses indicate in Python?", we should probably
show similar hesitation when it comes to giving ":" yet another
meaning.

This morning, I remembered a syntax idea I had a while back in
relation to lambda expressions that could perhaps be better applied to
assignment expressions, so I'll quickly recap the current options, and
then go on to discussing that. Since the threads are pretty sprawling,
I've also included a postscript going into more detail on my current
view of the pros and cons of the various syntax proposals presented so
far.

Expression first proposals:

    while (read_next_item() as value) is not None:
        ...

    while (read_next_item() -> value) is not None:
        ...

Target first proposal (current PEP):

    while (value := read_next_item()) is not None:
        ...

New keyword based target first proposal:

    while (value from read_next_item()) is not None:
        ...

The new one is a fairly arbitrary repurposing of the import system's
"from" keyword, but it avoids all the ambiguities of "as", is easier
to visually distinguish from other existing expression level keywords
than "as", avoids giving ":" yet another meaning, still lets us use a
keyword instead of a symbol, and gives the new expression type a more
clearly self-evident name ("from expressions", as opposed to the "from
statements" used for imports).

It also more easily lends itself to skipping over the details of the
defining expression when reading code aloud or in your head (e.g.
"while value is not None, where value comes from read_next_item()"
would be a legitimate way of reading the above for loop header, and
you could drop the trailing clause completely when the details aren't
particularly relevant). Avoiding the use of a colon as part of the
syntax also means that if we wanted to, we could potentially allow
optional type annotations in from-expressions ("target: annotation
from expression"), and even adopt them as a shorthand for the sentinel
pattern in function declarations (more on that below).

As far as the connection with "from module import name" goes, given
the proposed PEP 572 semantics, these three statements would all be
equivalent:

    from dotted.module import name
    name = __import__("dotted_module", fromlist=["name"]).name
    name from __import__("dotted_module", fromlist=["name"]).name

Other examples from the PEP:

    # Handle a matched regex
    if (match from pattern.search(data)) is not None:
        ...

    # Share a subexpression between a comprehension filter clause and its output
    filtered_data = [y for x in data if (y from f(x)) is not None]

    # Nested assignments
    assert 0 == (x from (y from (z from 0)))

    # Re-using fields in a container display
    stuff = [[y from f(x), x/y] for x in range(5)]

And the set/dict examples display where ":=" could be visually confusing:

    # Set display with local name bindings
    data = {
        value_a from 1,
        value_b from 2,
        value_c from 3,
    }

    # Dict display with local key & value name bindings
    data = {
        key_a from 'a': value_a from 1,
        key_b from 'b': value_b from 2,
        key_c from 'c': value_c from 3,
    }

Potential extension to simplifying the optional
non-shared-mutable-default-argument pattern:

     # Shared mutable default (stored directly in f.__defaults__)
     def f(shared = []):
         ....

     # Unshared mutable default (implementation details TBD)
     def f(unshared from []):
         ....

That last part would only be a potential extension beyond the scope of
PEP 572 (since it would go against the grain of "name = expression"
and "name from expression" otherwise being functionally equivalent in
their behaviour), but it's an opportunity that wouldn't arise if a
colon is part of the expression level name binding syntax.

Cheers,
Nick.

P.S. The pros and cons of the current syntax proposals, as I see them:

=== Expression first, 'as' keyword ===

    while (read_next_item() as value) is not None:
        ...

Pros:

  * typically reads nicely as pseudocode
  * "as" is already associated with namebinding operations

Cons:

  * syntactic ambiguity in with statement headers (major concern)
  * encourages a common misunderstanding of how with statements work
(major concern)
  * visual similarity between "as" and "and" makes name bindings easy to miss
  * syntactic ambiguity in except clause headers theoretically exists,
but is less of a concern due to the consistent type difference that
makes the parenthesised form pointless

=== Expression first, '->' symbol ===

    while (read_next_item() -> value) is not None:
        ...

Pros:

  * avoids the syntactic ambiguity of "as"
  * "->" is used for name bindings in at least some other languages
(but this is irrelevant to users for whom Python is their first, and
perhaps only, programming language)

Cons:

  * doesn't read like pseudocode (you need to interpret an arbitrary
non-arithmetic symbol)
  * invites the question "Why doesn't this use the 'as' keyword?"
  * symbols are typically harder to look up than keywords
  * symbols don't lend themselves to easy mnemonics
  * somewhat arbitrary repurposing of "->" compared to its use in
function annotations


=== Target first, ':=' symbol ===

    while (value := read_next_item()) is not None:
        ...

Pros:

  * avoids the syntactic ambiguity of "as"
  * being target first provides an obvious distinction from the "as" keyword
  * ":=" is used for name bindings in at least some other languages
(but this is irrelevant to users for whom Python is their first, and
perhaps only, language)

Cons:

  * symbols are typically harder to look up than keywords
  * symbols don't lend themselves to easy mnemonics
  * subject to a visual "line noise" phenomenon when combined with
other uses of ":" as a syntactic marker (e.g. slices, dict key/value
pairs, lambda expressions, type annotations)


=== Target first, 'from' keyword ===

    while (value from read_next_item()) is not None: # New
        ...

Pros:

  * avoids the syntactic ambiguity of "as"
  * being target first provides an obvious distinction from the "as" keyword
  * typically reads nicely as pseudocode
  * "from" is already associated with a namebinding operation ("from
module import name")

Cons:

  * I'm sure we'll think of some more, but all I have so far is that
the association with name binding is relatively weak and would need to
be learned

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia