Should this be considered a bug?
h = [1, 2, 3] d = dict(a=1, b=2) h += d # works h = h + d # exception -- Sent from my Android tablet with K-9 Mail. Please excuse my brevity.
On Sun, Oct 30, 2011 at 8:29 PM, Mike Meyer <mwm@mired.org> wrote:
No, the reason the latter fails is because it's unclear what the type of the result should be and the interpreter refuses to guess. In the former case, there's no such ambiguity (since the type of 'h' simply stays the same). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sun, Oct 30, 2011 at 9:10 PM, Georg Brandl <g.brandl@gmx.net> wrote:
It's still a bit inconsistent though (e.g. it doesn't work with tuple's __iadd__).
Tuple doesn't have an __iadd__, it's immutable :) As Antoine noted, list.__iadd__ is actually just list.extend under the hood, and accepts an arbitrary iterable. We're a bit more restrictive with other mutable container types (e.g. set, bytearray), but in those cases, the corresponding methods also only accept a limited selection of object types. (There's also a longstanding limitation in the type coercion system where returning NotImplemented doesn't work properly for the sequence slots at the C level - you have to use the numeric slots instead. That issue should finally be resolved in 3.3)
Again, that's an immutable type, and the various numeric types go to great lengths in order to play nicely together and negotiate the "correct" result type. Even there, we deliberately complain when we think something dodgy might be going on:
Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sun, 30 Oct 2011 20:55:29 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
While that's perfectly reasonable, it implies that some result type other than list is possible. I can't see what that would be since you can't add dictionaries. The only other iterable type you can add is tuples - and they don't display this behavior. So why not go ahead and fix this "bug" by making list.__add__ try converting the other object to a list, and adding a list.__radd__ with the same behavior? For that matter, a similar tweak for tuples wouldn't be inappropriate, and then the case of adding tuples & lists can raise a type error, paralleling the behavior of decimals and floats in the number hierarchy. <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/ Independent Software developer/SCM consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
On Mon, Oct 31, 2011 at 8:17 AM, Mike Meyer <mwm@mired.org> wrote:
The universe of container types is a lot bigger than the Python builtins. Suppose we made list.__add__ as permissive as list.__iadd__. Further suppose we make collections.deque.__add__ similarly permissive. Now "[] + deque() == []", but "deque() + [] == deque()". Contrast that with numeric types, which are designed to work together as a numeric tower, such that the type of a binary operation's result does *not* depend on the order of the operands: "1 + 1.1 == 2.1" and "1.1 + 1 == 2.1". Commutativity is an important expected property for addition and multiplication. Even NumPy's arrays respect that by using those operations for the element-wise equivalents - they use a separate method for matrix multiplication (which is not commutative by definition). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 30 October 2011 22:31, Nick Coghlan <ncoghlan@gmail.com> wrote:
The convention in mathematics is that addition is commutative but there is no such assumption for multiplication. In fact, multiplicative notation is commonly used for the composition law of non-commutative groups. Is there an established convention in computer languages? As for the commutativity of addition in python... well, string concatenation is not generally considered commutative :) -- Arnaud
On Mon, Oct 31, 2011 at 8:40 AM, Arnaud Delobelle <arnodel@gmail.com> wrote:
I was referring to the definition of multiplication in ordinary arithmetic (including complex numbers). Agreed that mathematicians in general are quite happy to use it for other relations that don't align with that definition.
As for the commutativity of addition in python... well, string concatenation is not generally considered commutative :)
Yeah, I was really talking about commutativity of result types rather than result values. The point about order mattering for values as soon as sequences get involved is a fair one (IIRC, that's one of the reasons set union uses '|' rather '+' - as a hint that the operation *is* commutative, even though container addition normally isn't). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Mon, 31 Oct 2011 08:31:18 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
True. But addition on lists isn't commutative. [1] + [2] != [2] + [1].
I agree. And if lists were commutative under addition, I'd say that was sufficient reason not to do this. But they aren't. <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/ Independent Software developer/SCM consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
On Mon, Oct 31, 2011 at 9:15 AM, Mike Meyer <mwm@mired.org> wrote:
Yeah, I was thinking in terms of type inference, but writing in terms of values. I should have stuck with my original examples, which showed the type inference rather than using specific instances. Basically, the design principle at work here is "the type of the result of a binary operation should not depend on the order of the operands". It has its roots in commutativity, but is not commutativity as such (since that is about values, not types). This principle drives lots of aspects of Python's implicit type conversion design. For example: 1. The left hand operand is typically given the first go at handling the operation, but is expected to return NotImplemented if it doesn't recognise the other operand 2. The right hand operand is given first go in the case where it is an instance of a *subclass* of the type of the left hand operand, thus allowing subclasses to consistently override the behaviour of the parent class 3. Most types (with the notable exception of numeric types) will only permit binary operations with other instances of the same type. For numeric types, the coercion is arranged so that the type of the result remains independent of the order of the operands. I did find an unfortunate case where Python 3 violates this design principle - bytes and bytearray objects accept any object that implements the buffer interface as the other operand, even in the __add__ case. This leads to the following asymmetries:
Now, the latter two cases are due to the problem I mentioned earlier where returning NotImplemented from sq_concat or sq_repeat doesn't work properly, but the bytes and bytearray interaction is exactly the kind of type asymmetry this guideline is intended to prevent. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Mon, Oct 31, 2011 at 9:46 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Interestingly, I found a similar ordering dependency exists for set() and frozenset():
So the status quo is less consistent than I previously thought. Regardless, it isn't enough to merely point out that the status quo is inconsistent. It's necessary to make a case for why the proposed change is sufficiently better to be worth the cost of making the change: http://www.boredomandlaziness.org/2011/02/status-quo-wins-stalemate.html Consider the "it's not worth the hassle" stodgy core developer response invoked ;) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Mon, 31 Oct 2011 10:20:35 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
I'll buy both arguments. Hopefully, someone is keeping a list of those inconsistencies (set vs. frozenset, etc.) to fix them at some point in the future. Thanks, <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/ Independent Software developer/SCM consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
On Sun, Oct 30, 2011 at 8:29 PM, Mike Meyer <mwm@mired.org> wrote:
No, the reason the latter fails is because it's unclear what the type of the result should be and the interpreter refuses to guess. In the former case, there's no such ambiguity (since the type of 'h' simply stays the same). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sun, Oct 30, 2011 at 9:10 PM, Georg Brandl <g.brandl@gmx.net> wrote:
It's still a bit inconsistent though (e.g. it doesn't work with tuple's __iadd__).
Tuple doesn't have an __iadd__, it's immutable :) As Antoine noted, list.__iadd__ is actually just list.extend under the hood, and accepts an arbitrary iterable. We're a bit more restrictive with other mutable container types (e.g. set, bytearray), but in those cases, the corresponding methods also only accept a limited selection of object types. (There's also a longstanding limitation in the type coercion system where returning NotImplemented doesn't work properly for the sequence slots at the C level - you have to use the numeric slots instead. That issue should finally be resolved in 3.3)
Again, that's an immutable type, and the various numeric types go to great lengths in order to play nicely together and negotiate the "correct" result type. Even there, we deliberately complain when we think something dodgy might be going on:
Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sun, 30 Oct 2011 20:55:29 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
While that's perfectly reasonable, it implies that some result type other than list is possible. I can't see what that would be since you can't add dictionaries. The only other iterable type you can add is tuples - and they don't display this behavior. So why not go ahead and fix this "bug" by making list.__add__ try converting the other object to a list, and adding a list.__radd__ with the same behavior? For that matter, a similar tweak for tuples wouldn't be inappropriate, and then the case of adding tuples & lists can raise a type error, paralleling the behavior of decimals and floats in the number hierarchy. <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/ Independent Software developer/SCM consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
On Mon, Oct 31, 2011 at 8:17 AM, Mike Meyer <mwm@mired.org> wrote:
The universe of container types is a lot bigger than the Python builtins. Suppose we made list.__add__ as permissive as list.__iadd__. Further suppose we make collections.deque.__add__ similarly permissive. Now "[] + deque() == []", but "deque() + [] == deque()". Contrast that with numeric types, which are designed to work together as a numeric tower, such that the type of a binary operation's result does *not* depend on the order of the operands: "1 + 1.1 == 2.1" and "1.1 + 1 == 2.1". Commutativity is an important expected property for addition and multiplication. Even NumPy's arrays respect that by using those operations for the element-wise equivalents - they use a separate method for matrix multiplication (which is not commutative by definition). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 30 October 2011 22:31, Nick Coghlan <ncoghlan@gmail.com> wrote:
The convention in mathematics is that addition is commutative but there is no such assumption for multiplication. In fact, multiplicative notation is commonly used for the composition law of non-commutative groups. Is there an established convention in computer languages? As for the commutativity of addition in python... well, string concatenation is not generally considered commutative :) -- Arnaud
On Mon, Oct 31, 2011 at 8:40 AM, Arnaud Delobelle <arnodel@gmail.com> wrote:
I was referring to the definition of multiplication in ordinary arithmetic (including complex numbers). Agreed that mathematicians in general are quite happy to use it for other relations that don't align with that definition.
As for the commutativity of addition in python... well, string concatenation is not generally considered commutative :)
Yeah, I was really talking about commutativity of result types rather than result values. The point about order mattering for values as soon as sequences get involved is a fair one (IIRC, that's one of the reasons set union uses '|' rather '+' - as a hint that the operation *is* commutative, even though container addition normally isn't). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Mon, 31 Oct 2011 08:31:18 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
True. But addition on lists isn't commutative. [1] + [2] != [2] + [1].
I agree. And if lists were commutative under addition, I'd say that was sufficient reason not to do this. But they aren't. <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/ Independent Software developer/SCM consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
On Mon, Oct 31, 2011 at 9:15 AM, Mike Meyer <mwm@mired.org> wrote:
Yeah, I was thinking in terms of type inference, but writing in terms of values. I should have stuck with my original examples, which showed the type inference rather than using specific instances. Basically, the design principle at work here is "the type of the result of a binary operation should not depend on the order of the operands". It has its roots in commutativity, but is not commutativity as such (since that is about values, not types). This principle drives lots of aspects of Python's implicit type conversion design. For example: 1. The left hand operand is typically given the first go at handling the operation, but is expected to return NotImplemented if it doesn't recognise the other operand 2. The right hand operand is given first go in the case where it is an instance of a *subclass* of the type of the left hand operand, thus allowing subclasses to consistently override the behaviour of the parent class 3. Most types (with the notable exception of numeric types) will only permit binary operations with other instances of the same type. For numeric types, the coercion is arranged so that the type of the result remains independent of the order of the operands. I did find an unfortunate case where Python 3 violates this design principle - bytes and bytearray objects accept any object that implements the buffer interface as the other operand, even in the __add__ case. This leads to the following asymmetries:
Now, the latter two cases are due to the problem I mentioned earlier where returning NotImplemented from sq_concat or sq_repeat doesn't work properly, but the bytes and bytearray interaction is exactly the kind of type asymmetry this guideline is intended to prevent. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Mon, Oct 31, 2011 at 9:46 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Interestingly, I found a similar ordering dependency exists for set() and frozenset():
So the status quo is less consistent than I previously thought. Regardless, it isn't enough to merely point out that the status quo is inconsistent. It's necessary to make a case for why the proposed change is sufficiently better to be worth the cost of making the change: http://www.boredomandlaziness.org/2011/02/status-quo-wins-stalemate.html Consider the "it's not worth the hassle" stodgy core developer response invoked ;) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Mon, 31 Oct 2011 10:20:35 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
I'll buy both arguments. Hopefully, someone is keeping a list of those inconsistencies (set vs. frozenset, etc.) to fix them at some point in the future. Thanks, <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/ Independent Software developer/SCM consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
participants (6)
-
Antoine Pitrou -
Arnaud Delobelle -
Carl M. Johnson -
Georg Brandl -
Mike Meyer -
Nick Coghlan