[ python-Bugs-1306777 ] Augmented assigment to mutable objects in tuples fail

[SourceForge.net]

Bugs item #1306777, was opened at 2005-09-28 12:59
Message generated for change (Comment added) made by yorick
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1306777&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.4
Status: Closed
Resolution: Invalid
Priority: 5
Submitted By: Mattias Engdegård (yorick)
Assigned to: Nobody/Anonymous (nobody)
Summary: Augmented assigment to mutable objects in tuples fail

Initial Comment:
>>> t=(set([2]),)
>>> t[0] |= set([7])
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: object does not support item assignment

but the contained set is mutable, and in fact:
>>>  t[0].__ior__(set([7]))
set([2, 7])
>>> t
(set([2, 7]),)

If I use a mutable container (a list) in the first
case, it works:
>>> u=[set([2])]
>>> u[0] |= set([7])
>>> u
[set([2, 7])]

But note that the list has not been mutated - only the
set, so there would be no need for a mutable container.
This is highly counter-intuitive - augmented assigment
should do in-place operations on mutable types (which
it does) and should therefore pose no restriction on
the mutability of the container (which fails).


----------------------------------------------------------------------

>Comment By: Mattias Engdegård (yorick)
Date: 2005-09-28 18:22

Message:
Logged In: YES 
user_id=432579

Then maybe the language definition is in error since its
consequences are counter-intuitive and impractical. Would
any reasonable code break if it was modified to what a naïve
user would expect? For instance, the conditional writeback
suggested earlier.


----------------------------------------------------------------------

Comment By: Phillip J. Eby (pje)
Date: 2005-09-28 16:27

Message:
Logged In: YES 
user_id=56214

This is not a bug.  The language-defined behavior of
augmented assignment is that:

  lvalue |= rvalue

translates to:

  lvalue = lvalue.__ior__(rvalue)

Since in this example the lvalue cannot be assigned to, it
is your code that has the bug. Note that some objects'
__ior__ method may not return the same object!  Therefore,
using a tuple to store the target of an in-place update
would only work with objects whose __ior__ returns self.  If
you know that this will always be the case in your code,
then simply do this:

  ts = t[0]
  ts |= set([7])

Again, note that this means that if t[0].__ior__ does not
return self, then ts will not be the same object as t[0],
leaving the tuple un-updated.  In short, you can't use
augmented assignments to tuple items in the general case.

Note, by the way, that sets have other methods for in-place
updating, and your code would probably be clearer using one
of those, since no lvalue confusion arises.  The whole
purpose of augmented assignment is to deal with the
possibility that an in-place update might or might not
result in the same object. If an object offers a method that
unequivocally mutates it, your code will be clearer using that.


----------------------------------------------------------------------

Comment By: Reinhold Birkenfeld (birkenfeld)
Date: 2005-09-28 15:13

Message:
Logged In: YES 
user_id=1188172

I don't know. This way, __setitem__ would not be called even
if it exists. That may pose b/w compatibility problems. I'm
asking python-dev.

----------------------------------------------------------------------

Comment By: Mattias Engdegård (yorick)
Date: 2005-09-28 14:52

Message:
Logged In: YES 
user_id=432579

Certainly, but I meant that we could emit bytecode to
compare the result of INPLACE_OR and do a conditional writeback.


----------------------------------------------------------------------

Comment By: Reinhold Birkenfeld (birkenfeld)
Date: 2005-09-28 14:41

Message:
Logged In: YES 
user_id=1188172

The bytecode generation happens before any code is executed,
so the generated bytecode for x |= y is always the same
(except, perhaps, when constants are involved).

----------------------------------------------------------------------

Comment By: Mattias Engdegård (yorick)
Date: 2005-09-28 14:24

Message:
Logged In: YES 
user_id=432579

Thank you for your analysis.
I'm not intimitely familiar with the bytecodes, but one way
would be to omit the writeback (STORE_SUBSCR) if the result
of INPLACE_OR is identical to its input. This could probably
be done without changing the bytecodes but might profit from
some changes for speed and compactness.

That is (pseudocode):
_t1 = t[i]
_t2 = inplace_or(_t1, a)
if _t2 is not _t1:
    t[i] = _t2

Another variant would be to add indexed variants of the
augmented assigment methods; that is, augmented variants of
__setitem__, but that has other drawbacks. However, it might
be useful in its own regard in some cases.


----------------------------------------------------------------------

Comment By: Reinhold Birkenfeld (birkenfeld)
Date: 2005-09-28 14:17

Message:
Logged In: YES 
user_id=1188172

Generally there are two possibilites for the __ixxx__ methods:
1) Modify self and return self
2) Create a new instance with desired attributes and return it
    (necessary for e.g. integers)

The second case cannot be handled by immutable containers.

Hmm, maybe PySequence_SetItem should check whether the
assigned item is already there and then succeed.

Attaching a minimal patch for PySequence_SetItem (not sure
about PyObject_SetItem).

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2005-09-28 13:50

Message:
Logged In: YES 
user_id=6656

Yuck, I agree that that's pretty icky.  But the disassembly makes things 
clear:

>>> dis.dis(compile('t[0] |= a', '', 'single'))
  1           0 LOAD_NAME                0 (t)
              3 LOAD_CONST               0 (0)
              6 DUP_TOPX                 2
              9 BINARY_SUBSCR       
             10 LOAD_NAME                1 (a)
             13 INPLACE_OR          
             14 ROT_THREE           
             15 STORE_SUBSCR        
             16 LOAD_CONST               1 (None)
             19 RETURN_VALUE        

In fact...

>>> s = set([1])
>>> t = (s,)
>>> t[0] |= set([2])
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: object does not support item assignment
>>> s
set([1, 2])
>>> 

Oof.

Not sure what to do about this.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1306777&group_id=5470