[Python-Dev] PEP 572: Assignment Expressions

[Terry Reedy]

On 4/17/2018 3:46 AM, Chris Angelico wrote:

> Abstract
> ========
> 
> This is a proposal for creating a way to assign to names within an expression.

I started at -something as this is nice but not necessary.  I migrated 
to +something for the specific, limited proposal you wrote above: 
expressions of the form "name := expression".

> Additionally, the precise scope of comprehensions is adjusted, to maintain
> consistency and follow expectations.

We fiddled with comprehension scopes, and broke some code, in 3.0.  I 
oppose doing so again.  People expect their 3.x code to continue working 
in future versions.  Breaking that expectation should require 
deprecation for at least 2 versions.

> Rationale
> =========
> 
> Naming the result of an expression is an important part of programming,
> allowing a descriptive name to be used in place of a longer expression,
> and permitting reuse.

Right.  In other words, "name := expression".

> Merely introducing a way to assign as an expression
> would create bizarre edge cases around comprehensions, though, and to avoid
> the worst of the confusions, we change the definition of comprehensions,
> causing some edge cases to be interpreted differently, but maintaining the
> existing behaviour in the majority of situations.

If it is really true that introducing 'name := expression' requires such 
a side-effect, then I might oppose it.

> 
> Syntax and semantics
> ====================
> 
> In any context where arbitrary Python expressions can be used, a **named
> expression** can appear. This is of the form ``target := expr`` where
> ``expr`` is any valid Python expression, and ``target`` is any valid
> assignment target.

This generalization is different from what you said in the abstract and 
rationale.  No rationale is given.  After reading Nick's examination of 
the generalization, and your response, -1.

> The value of such a named expression is the same as the incorporated
> expression, with the additional side-effect that the target is assigned
> that value::

As someone else noted, you only use names as targets, thus providing no 
rationale for anything else.

>      # Handle a matched regex
>      if (match := pattern.search(data)) is not None:
>          ...
> 
>      # A more explicit alternative to the 2-arg form of iter() invocation
>      while (value := read_next_item()) is not None:
>          ...

To me, being able to name and test expressions fits with Python names 
not being typed.  To me, usage such as the above is the justification 
for the limited proposal.

>      # Share a subexpression between a comprehension filter clause and its output
>      filtered_data = [y for x in data if (y := f(x)) is not None]

And this is secondary.

> Differences from regular assignment statements
> ----------------------------------------------
> 
> Most importantly, since ``:=`` is an expression, it can be used in contexts
> where statements are illegal, including lambda functions and comprehensions.
> 
> An assignment statement can assign to multiple targets, left-to-right::
> 
>      x = y = z = 0

This is a bad example as there is very seldom a reason to assign 
multiple names, as opposed to multiple targets.  Here is a typical real 
example.

     self.x = x = expression
     # Use local x in the rest of the method.

In "x = y = 0", x and y likely represent two *different* concepts 
(variables) that happen to be initialized with the same value.  One 
could instead write "x,y = 0,0".

> The equivalent assignment expression

should be a syntax error.

> is parsed as separate binary operators,

':=' is not a binary operator, any more than '=' is, as names, and 
targets in general, are not objects.  Neither fetch and operate on the 
current value, if any, of the name or target.  Therefore neither has an 
'associativity'.

> and is therefore processed right-to-left, as if it were spelled thus::
> 
>      assert 0 == (x := (y := (z := 0)))

Parentheses should be required, to maintain the syntax "name := expression".

> Augmented assignment is not supported in expression form::
> 
>      >>> x +:= 1
>        File "<stdin>", line 1
>          x +:= 1
>              ^
>      SyntaxError: invalid syntax

I would have expected :+=, but agree with the omission.

> Otherwise, the semantics of assignment are identical in statement and
> expression forms.

Mostly replacing '=' with ':=' is a different proposal and a different 
goal than naming expressions within an expression for reuse (primarily) 
within the expression (including compound expressions).

Proposing an un-augmentable, un-chainable, name_only := expression 
expression would not be duplicating assignment statements.

> Alterations to comprehensions
> -----------------------------
> 
> The current behaviour of list/set/dict comprehensions and generator
> expressions has some edge cases that would behave strangely if an assignment
> expression were to be used.

You have not shown this.  Your examples do not involve assignment 
expressions, and adding them should make no difference.  Changing the 
scoping of comprehensions should be a separate PEP.

  Therefore the proposed semantics are changed,
> removing the current edge cases, and instead altering their behaviour *only*
> in a class scope.
> 
> As of Python 3.7, the outermost iterable of any comprehension is evaluated
> in the surrounding context, and then passed as an argument to the implicit
> function that evaluates the comprehension.
> 
> Under this proposal, the entire body of the comprehension is evaluated in
> its implicit function. Names not assigned to within the comprehension are
> located in the surrounding scopes, as with normal lookups. As one special
> case, a comprehension at class scope will **eagerly bind** any name which
> is already defined in the class scope.
> 
> A list comprehension can be unrolled into an equivalent function. With
> Python 3.7 semantics::
> 
>      numbers = [x + y for x in range(3) for y in range(4)]
>      # Is approximately equivalent to
>      def <listcomp>(iterator):
>          result = []
>          for x in iterator:
>              for y in range(4):
>                  result.append(x + y)
>          return result
>      numbers = <listcomp>(iter(range(3)))
> 
> Under the new semantics, this would instead be equivalent to::
> 
>      def <listcomp>():
>          result = []
>          for x in range(3):
>              for y in range(4):
>                  result.append(x + y)
>          return result
>      numbers = <listcomp>()

Why make the change?

> When a class scope is involved, a naive transformation into a function would
> prevent name lookups (as the function would behave like a method)::
> 
>      class X:
>          names = ["Fred", "Barney", "Joe"]
>          prefix = "> "
>          prefixed_names = [prefix + name for name in names]
> 
> With Python 3.7 semantics,

I believe in all of 3.x ..

> this will evaluate the outermost iterable at class
> scope, which will succeed; but it will evaluate everything else in a function::
> 
>      class X:
>          names = ["Fred", "Barney", "Joe"]
>          prefix = "> "
>          def <listcomp>(iterator):
>              result = []
>              for name in iterator:
>                  result.append(prefix + name)
>              return result
>          prefixed_names = <listcomp>(iter(names))
> 
> The name ``prefix`` is thus searched for at global scope, ignoring the class
> name.

And today it fails.  This has nothing to do with adding name assignment 
expressions.

> Under the proposed semantics, this name will be eagerly bound; and the
> same early binding then handles the outermost iterable as well. The list
> comprehension is thus approximately equivalent to::
> 
>      class X:
>          names = ["Fred", "Barney", "Joe"]
>          prefix = "> "
>          def <listcomp>(names=names, prefix=prefix):
>              result = []
>              for name in names:
>                  result.append(prefix + name)
>              return result
>          prefixed_names = <listcomp>()
> 
> With list comprehensions, this is unlikely to cause any confusion. With
> generator expressions, this has the potential to affect behaviour, as the
> eager binding means that the name could be rebound between the creation of
> the genexp and the first call to ``next()``. It is, however, more closely
> aligned to normal expectations.  The effect is ONLY seen with names that
> are looked up from class scope; global names (eg ``range()``) will still
> be late-bound as usual.
> 
> One consequence of this change is that certain bugs in genexps will not
> be detected until the first call to ``next()``, where today they would be
> caught upon creation of the generator. See 'open questions' below.
> 
> 
> Recommended use-cases
> =====================
> 
> Simplifying list comprehensions
> -------------------------------

I consider this secondary and would put it second.

> Capturing condition values
> --------------------------

I would put this first, as you did above.

> Assignment expressions can be used to good effect in the header of
> an ``if`` or ``while`` statement::
> 
>      # Proposed syntax
>      while (command := input("> ")) != "quit":
>          print("You entered:", command)
> 
>      # Capturing regular expression match objects
>      # See, for instance, Lib/pydoc.py, which uses a multiline spelling
>      # of this effect
>      if match := re.search(pat, text):
>          print("Found:", match.group(0))
> 
>      # Reading socket data until an empty string is returned
>      while data := sock.read():
>          print("Received data:", data)
> 
>      # Equivalent in current Python, not caring about function return value
>      while input("> ") != "quit":
>          print("You entered a command.")
> 
>      # To capture the return value in current Python demands a four-line
>      # loop header.
>      while True:
>          command = input("> ");
>          if command == "quit":
>              break
>          print("You entered:", command)

This idiom is not obvious to beginners and is awkward at best, so I 
consider eliminating this the biggest gain.  Beginners commonly write 
little games and entry loops and get tripped up trying to do so.

> Particularly with the ``while`` loop, this can remove the need to have an
> infinite loop, an assignment, and a condition. It also creates a smooth
> parallel between a loop which simply uses a function call as its condition,
> and one which uses that as its condition but also uses the actual value.

...

Bottom line: I suggest rewriting again, as indicated, changing title to 
'Name Assignment Expressions'.

-- 
Terry Jan Reedy