[Python-Dev] behavior of inplace operations

[David Abrahams]

My initial post asking about the implementation of += sparked a small
thread over on python-list, from which I've come to the conclusion that my
little optimization suggestion (don't try to set the attribute or item if
the inplace op returns its first argument) is actually more semantically
correct.

For better or worse, these ideas aren't all mine, as
http://aspn.activestate.com/ASPN/Mail/Message/python-list/1222524 attests.

Consider:

>>> t = ([1],[2],[3])
>>> t[0].append(2) # OK, the elements of the tuple are mutable
>>> t
([1, 2], [2], [3])
>>>
>>> t[1] += [3]    # ?? Just did the equivalent operation above
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: object doesn't support item assignment
>>> t # Despite the exception, the operation succeeded!
([1, 2], [2, 3], [3])

So, the exception happens because the user is ostensibly trying to modify
this immutable tuple... but of course she's not. She's just trying to
modify the element of the tuple, which is itself mutable, and that makes
the exception surprising. Even more surprising in light of the exception is
the fact that everything seems to have worked. In order to get this all to
make sense, she needs to twist her brain into remembering that inplace
operations don't really just modify their targets "in place", but also try
to "replace" them.

However, if we just set up the inplace operations so that when they return
the original object there's no "replace", all of these problems go away. We
don't lose any safety; trying to do += on an immutable tuple element will
still fail. Also it makes tuples a generic replacement for lists in more
places.

There are other, more-perverse cases which the proposed change in semantics
would also fix. For example:

>>> class X(object):
...     def __init__(self, l):
...             self.container = l # will form a cycle
...             self.stuff  = []
...     def __iadd__(self, other):
...             self.stuff += other # add to X's internal list
...             del self.container[0]
...             return self
...
>>> l = [ 1, 2, 3]
>>> l.append(X(l))                # the X element refers to l
>>> l
[1, 2, 3, <__main__.X object at 0x00876668>]
>>> l[3] += 'a'     # the element is gone by write-back time.
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
IndexError: list assignment index out of range
>>> l # But everything succeeded
[2, 3, <__main__.X object at 0x00876668>]
>>> l[2].stuff
['a']
>>> l.append('tail') # this one's even wierder
>>> l[2] += 'a'
>>> l
[3, <__main__.X object at 0x00876668>, <__main__.X object at 0x00876668>]

These are too esoteric to be compelling on their own, but my proposal would
also make them work as expected.

Thoughts?
-Dave

------------

P.S. This error message is kind of wierd:

>>> t = ([1],[2],[3])
>>> t[1] += 3
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: argument to += must be iterable
                          ^^^^^^^^^^^^^^^^ ??

+---------------------------------------------------------------+
                  David Abrahams
      C++ Booster (http://www.boost.org)               O__  ==
      Pythonista (http://www.python.org)              c/ /'_ ==
  resume: http://users.rcn.com/abrahams/resume.html  (*) \(*) ==
          email: david.abrahams@rcn.com
+---------------------------------------------------------------+