On Oct 30, 2011, at 12:55 AM, Nick Coghlan wrote:

On Sun, Oct 30, 2011 at 8:29 PM, Mike Meyer wrote:

...

h = [1, 2, 3] d = dict(a=1, b=2) h += d # works h = h + d # exception

No, the reason the latter fails is because it's unclear what the type of the result should be and the interpreter refuses to guess. In the former case, there's no such ambiguity (since the type of 'h' simply stays the same).

What about the fact that dictionaries have no particular order? If you're expecting my_list += my_set to come out in a particular order (a common rookie mistake when dealing with dicts as well), you can be silently bitten.

On Sun, 30 Oct 2011 12:10:39 +0100 Georg Brandl wrote:

On 10/30/2011 11:55 AM, Nick Coghlan wrote:

...

On Sun, Oct 30, 2011 at 8:29 PM, Mike Meyer wrote:

...

h = [1, 2, 3] d = dict(a=1, b=2) h += d # works h = h + d # exception

No, the reason the latter fails is because it's unclear what the type of the result should be and the interpreter refuses to guess. In the former case, there's no such ambiguity (since the type of 'h' simply stays the same).

It's still a bit inconsistent though (e.g. it doesn't work with tuple's __iadd__).

I think list.__iadd__ is simply the same as list.extend, which takes an arbitrary iterable (and therefore accepts dicts, sets, etc.). Regards Antoine.

On 10/30/2011 01:17 PM, Nick Coghlan wrote:

On Sun, Oct 30, 2011 at 9:10 PM, Georg Brandl wrote:

...

It's still a bit inconsistent though (e.g. it doesn't work with tuple's __iadd__).

Tuple doesn't have an __iadd__, it's immutable :)

Ah yes, that might be the reason ;)

As Antoine noted, list.__iadd__ is actually just list.extend under the hood, and accepts an arbitrary iterable.

We're a bit more restrictive with other mutable container types (e.g. set, bytearray), but in those cases, the corresponding methods also only accept a limited selection of object types.

Makes sense. Georg

On Mon, Oct 31, 2011 at 8:17 AM, Mike Meyer wrote:

The only other iterable type you can add is tuples - and they don't display this behavior. So why not go ahead and fix this "bug" by making list.__add__ try converting the other object to a list, and adding a list.__radd__ with the same behavior?

For that matter, a similar tweak for tuples wouldn't be inappropriate, and then the case of adding tuples & lists can raise a type error, paralleling the behavior of decimals and floats in the number hierarchy.

The universe of container types is a lot bigger than the Python builtins. Suppose we made list.__add__ as permissive as list.__iadd__. Further suppose we make collections.deque.__add__ similarly permissive. Now "[] + deque() == []", but "deque() + [] == deque()". Contrast that with numeric types, which are designed to work together as a numeric tower, such that the type of a binary operation's result does *not* depend on the order of the operands: "1 + 1.1 == 2.1" and "1.1 + 1 == 2.1". Commutativity is an important expected property for addition and multiplication. Even NumPy's arrays respect that by using those operations for the element-wise equivalents - they use a separate method for matrix multiplication (which is not commutative by definition). Cheers, Nick. -- Nick Coghlan   |   ncoghlan@gmail.com   |   Brisbane, Australia

On Mon, Oct 31, 2011 at 8:40 AM, Arnaud Delobelle wrote:

The convention in mathematics is that addition is commutative but there is no such assumption for multiplication.  In fact, multiplicative notation is commonly used for the composition law of non-commutative groups.

Is there an established convention in computer languages?

I was referring to the definition of multiplication in ordinary arithmetic (including complex numbers). Agreed that mathematicians in general are quite happy to use it for other relations that don't align with that definition.

As for the commutativity of addition in python... well, string concatenation is not generally considered commutative :)

Yeah, I was really talking about commutativity of result types rather than result values. The point about order mattering for values as soon as sequences get involved is a fair one (IIRC, that's one of the reasons set union uses '|' rather '+' - as a hint that the operation *is* commutative, even though container addition normally isn't). Cheers, Nick. -- Nick Coghlan   |   ncoghlan@gmail.com   |   Brisbane, Australia

On Mon, Oct 31, 2011 at 9:15 AM, Mike Meyer wrote:

...

Commutativity is an important expected property for addition and multiplication. Even NumPy's arrays respect that by using those operations for the element-wise equivalents - they use a separate method for matrix multiplication (which is not commutative by definition).

I agree. And if lists were commutative under addition, I'd say that was sufficient reason not to do this. But they aren't.

Yeah, I was thinking in terms of type inference, but writing in terms of values. I should have stuck with my original examples, which showed the type inference rather than using specific instances. Basically, the design principle at work here is "the type of the result of a binary operation should not depend on the order of the operands". It has its roots in commutativity, but is not commutativity as such (since that is about values, not types). This principle drives lots of aspects of Python's implicit type conversion design. For example: 1. The left hand operand is typically given the first go at handling the operation, but is expected to return NotImplemented if it doesn't recognise the other operand 2. The right hand operand is given first go in the case where it is an instance of a *subclass* of the type of the left hand operand, thus allowing subclasses to consistently override the behaviour of the parent class 3. Most types (with the notable exception of numeric types) will only permit binary operations with other instances of the same type. For numeric types, the coercion is arranged so that the type of the result remains independent of the order of the operands. I did find an unfortunate case where Python 3 violates this design principle - bytes and bytearray objects accept any object that implements the buffer interface as the other operand, even in the __add__ case. This leads to the following asymmetries:

...

...

b'' + bytearray() b'' b'' + memoryview(b'') b'' bytearray() + b'' bytearray(b'') bytearray() + memoryview(b'') bytearray(b'') memoryview(b'') + b'' Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for +: 'memoryview' and 'bytes' memoryview(b'') + bytearray(b'') Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for +: 'memoryview' and 'bytearray'

Now, the latter two cases are due to the problem I mentioned earlier where returning NotImplemented from sq_concat or sq_repeat doesn't work properly, but the bytes and bytearray interaction is exactly the kind of type asymmetry this guideline is intended to prevent. Cheers, Nick. -- Nick Coghlan   |   ncoghlan@gmail.com   |   Brisbane, Australia

On Mon, 31 Oct 2011 10:20:35 +1000 Nick Coghlan wrote:

On Mon, Oct 31, 2011 at 9:46 AM, Nick Coghlan wrote: So the status quo is less consistent than I previously thought.

Regardless, it isn't enough to merely point out that the status quo is inconsistent. It's necessary to make a case for why the proposed change is sufficiently better to be worth the cost of making the change: http://www.boredomandlaziness.org/2011/02/status-quo-wins-stalemate.html

Consider the "it's not worth the hassle" stodgy core developer response invoked ;)

I'll buy both arguments. Hopefully, someone is keeping a list of those inconsistencies (set vs. frozenset, etc.) to fix them at some point in the future. Thanks, http://www.mired.org/ Independent Software developer/SCM consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org