[Python-ideas] Augmented assignment [was Re: Adding "+" and "+=" operators to dict]

[Andrew Barnert]

On Feb 14, 2015, at 17:30, Steven D'Aprano <steve at pearwood.info> wrote:

> This sub-thread has long since drifted away from dicts, so I've changed 
> the subject to make that clear.
> 
> On Sat, Feb 14, 2015 at 11:41:52AM -0800, Chris Barker wrote:
> 
>> The fact that you can't directly use augmented assignment on an object
>> contained in an immutable is not a bug, but it certainly is a wart --
>> particuarly since it will raise an Exception AFTER it has, in fact,
>> performed the operation requested.
> 
> Yes, but you can use augmented assignment on an object contained in an 
> immutable under some circumstances. Here are three examples 
> demonstrating outright failure, weird super-position of 
> failed-but-succeeded, and success.
> 
> 
> py> t = (1, )
> py> t[0] += 1
> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
> TypeError: 'tuple' object does not support item assignment
> py> t
> (1,)
> 
> py> t = ([1], )
> py> t[0] += [1]
> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
> TypeError: 'tuple' object does not support item assignment
> py> t
> ([1, 1],)
> 
> py> t[0][0] += 1
> py> t
> ([2, 1],)
> 
> 
>> I have argued that this never would have come up if augmented assignment
>> were only used for in-place operations,
> 
> 
> And it would never happen if augmented assignment *never* was used for 
> in-place operations. If it always required an assignment, then if the 
> assignment failed, the object in question would be unchanged.
> 
> Alas, there's no way to enforce the rule that __iadd__ doesn't modify 
> objects in place, and it actually is a nice optimization when they can 
> do so.

No, it's not just a nice optimization, it's an important part of the semantics. The whole point of being able to share mutable objects is being able to mutate them in a way that all the sharers see.

If __iadd__ didn't modify objects in-place, you'd have this:

    py> a, b = [], []
    py> c = [a, b]
    py> c[1] += [1]
    py> b
    []

Of course this would make perfect sense if Python were a pure immutable language, or a lower-level language like C++ where c = [a, b] made copies of a and b rather than creating new names referring to the same lists, or if Python had made the decision long ago that mutation is always spelled with an explicit method call rather than operator-like syntax. And any of those three alternatives could be part of a good language. But that language would be substantially different from Python.

>> I don't know enough about how this all works under the hood to know if it
>> could be made to work, but it seems the intention is clear here:
>> 
>> object[index] += something.
>> 
>> 
>> is a shorthand for:
>> 
>> tmp = object[index]
>> tmp += something
> 
> No, that doesn't work. If tmp is *immutable*, then object[index] never 
> gets updated.
> 
> The intention as I understand it is that:
> 
> reference += expr
> 
> should be syntactic sugar (possibly optimized) for:
> 
> reference = reference + expr
> 
> where "reference" means (for example) bare names, item references, key 
> references, and chaining the same:
> 
> n
> n[0]
> n[0]['spam']
> n[0]['spam'].eggs
> 
> etc.
> 
> I wonder if we can make this work more clearly if augmented assignments 
> checked whether the same object is returned and skipped the assignment 
> in that case?

I already answered that earlier. There are plenty of objects that necessarily rely on the assumption that item/attr assignment always means __setitem__/__setattr__.

Consider any object with a cache that has to be invalidated when one of its members or attributes is set. Or a sorted list that may have to move an element if it's replaced. Or really almost any case where a custom __setattr__, an @property or custom descriptor, or a non-trivial __setitem__ is useful. All of there would break.

What you're essentially proposing is that augmented assignment is no longer really assignment, so classes that want to manage assignment in some way can't manage augmented assignment.