dict '+' operator and slicing support for pop
First of all, this is my first attempt at submitting an idea to python-ideas. So, here it goes. :) 1. Add ability to use '+' operator for dicts I often wonder why list and tuple instances have '+' and '+=' operators but dicts don't? It's not that rare in my code (and code written by others, as it seems) that i have to write: a.update(b) return a I do understand that adding additional magic method may be inappropriate for dict, but I think it would be nice addition to a language. So, my proposal is that: x = a + b would become equivalent to x = dict(a, **b) a += b would become equivalent to a.update(b) And the example I gave before would be translated to: return a + b Note, that there is a difference between these two examples in semantics, the latter one creates a new dict. But that's what user doesn't care about in 99% of use-cases. A very basic implementation in Python:
class Dict(dict): ... def __add__(self, other): ... return self.__class__(self, **other) ... def __iadd__(self, other): ... self.update(other) ... return self ... a = Dict(foo=12, bar=14, baz=16) b = Dict(spam=13, eggs=17, bacon=19) a + b {'bar': 14, 'spam': 13, 'bacon': 19, 'eggs': 17, 'foo': 12, 'baz': 16} a += b a {'bar': 14, 'spam': 13, 'bacon': 19, 'eggs': 17, 'foo': 12, 'baz': 16} b {'eggs': 17, 'bacon': 19, 'spam': 13}
class Dict(dict): ... def __add__(self, other): ... return self.__class__(self, **other) ... def __iadd__(self, other): ... self.update(other) ... return self ... a = Dict(foo=12, bar=14, baz=16) b = Dict(spam=13, eggs=17, bacon=19) a + b {'bar': 14, 'spam': 13, 'bacon': 19, 'eggs': 17, 'foo': 12, 'baz': 16} a += b a {'bar': 14, 'spam': 13, 'bacon': 19, 'eggs': 17, 'foo': 12, 'baz': 16} b {'eggs': 17, 'bacon': 19, 'spam': 13}
Note: if this is ever going to be implemented, then Mapping ABCs will have to implement these methods, which doesn't sound like backwards-compatible to me. :) 2. Ability to use slices in pop This was discussed earlier (4.5 years ago, actually) in this thread: http://mail.python.org/pipermail/python-dev/2004-November/049895.html Even though original request became a full-blown proposal (PEP-3132) and was implemented in py3k, the internal discussion about pop allowing slice as arguments has silenced. There was some positive feedback from Python developers and I think I can provide a patch for this functionality in 2 weeks. Is there still some interest in this? There is nothing really hard, this would be something like this:
class List(list): ... def pop(self, index_or_slice): ... ret = self[index_or_slice] ... del self[index_or_slice] ... return ret ... x = List(range(10)) x.pop(slice(1, 4)) [1, 2, 3] x [0, 4, 5, 6, 7, 8, 9] x.pop(5) 8 x [0, 4, 5, 6, 7, 9]
Note: some people think that pop returning different list or item depending on what is being passed to pop() is bad or something. I don't see a problem here, because simple some_list[index_or_slice] can also return list or just one item depending on what type `index_or_slice` is. -- Wbr, Andrii V. Mishkovskyi. He's got a heart of a little child, and he keeps it in a jar on his desk.
On Tue, Mar 17, 2009 at 11:00 AM, Andrii V. Mishkovskyi <mishok13@gmail.com> wrote:
1. Add ability to use '+' operator for dicts
I often wonder why list and tuple instances have '+' and '+=' operators but dicts don't? It's not that rare in my code (and code written by others, as it seems) that i have to write:
a.update(b) return a
I do understand that adding additional magic method may be inappropriate for dict, but I think it would be nice addition to a language. So, my proposal is that:
x = a + b would become equivalent to x = dict(a, **b)
a += b would become equivalent to a.update(b)
That's one way to define dict addition but it's not the only, or even, the best one. It's hard to put in words exactly why but I expect "a+b" to take into account the full state of the operands, not just a part of it. In your proposal the values of the first dict for the common keys are effectively ignored, which doesn't seem to me as a good fit for an additive operation. I would find at least as reasonable and intuitive the following definition that doesn't leak information: def sum_dicts(*dicts): from collections import defaultdict s = defaultdict(list) for d in dicts: for k,v in d.iteritems(): s[k].append(v) return s
d1 = {'a':2,'b':5} d2 = {'a':2,'c':6,'z':3} d3 = {'b':2,'c':5} sum_dicts(d1,d2,d3) defaultdict(<type 'list'>, {'a': [2, 2], 'c': [6, 5], 'b': [5, 2], 'z': [3]})
George
On Tue, Mar 17, 2009 at 9:03 AM, George Sakkis <george.sakkis@gmail.com> wrote:
On Tue, Mar 17, 2009 at 11:00 AM, Andrii V. Mishkovskyi <mishok13@gmail.com> wrote:
1. Add ability to use '+' operator for dicts
I often wonder why list and tuple instances have '+' and '+=' operators but dicts don't? It's not that rare in my code (and code written by others, as it seems) that i have to write:
a.update(b) return a
I do understand that adding additional magic method may be inappropriate for dict, but I think it would be nice addition to a language. So, my proposal is that:
x = a + b would become equivalent to x = dict(a, **b)
a += b would become equivalent to a.update(b)
That's one way to define dict addition but it's not the only, or even, the best one. It's hard to put in words exactly why but I expect "a+b" to take into account the full state of the operands, not just a part of it. In your proposal the values of the first dict for the common keys are effectively ignored, which doesn't seem to me as a good fit for an additive operation. I would find at least as reasonable and intuitive the following definition that doesn't leak information:
def sum_dicts(*dicts): from collections import defaultdict s = defaultdict(list) for d in dicts: for k,v in d.iteritems(): s[k].append(v) return s
d1 = {'a':2,'b':5} d2 = {'a':2,'c':6,'z':3} d3 = {'b':2,'c':5} sum_dicts(d1,d2,d3) defaultdict(<type 'list'>, {'a': [2, 2], 'c': [6, 5], 'b': [5, 2], 'z': [3]})
Both of the ideas suffer from "+ is no longer commutative", which sort-of bothers me. I say sort-of, because I would actually prefer Andrii's semantics over yours, and if you prefer the elements from b, you use 'a + b', but if you prefer the elements from a, you use 'b + a'. Then again, I'm tending towards a -.75 on the entire idea; despite it being convenient, I can see non-comutativity as being confusing. As for the list slice popping...I'm tending towards a -1. While I can see the convenience in some cases, I'm just not sure it's compelling enough (especially because you need to generate the slice in advance of using it). As stated in the past...not all 2 line functions need to be built-in or syntax. I don't believe either of these are able to pass the "is it necessary as part of a compelling use-case?" question. - Josiah
On Tue, 17 Mar 2009, Josiah Carlson wrote:
On Tue, Mar 17, 2009 at 9:03 AM, George Sakkis <george.sakkis@gmail.com> wrote:
On Tue, Mar 17, 2009 at 11:00 AM, Andrii V. Mishkovskyi <mishok13@gmail.com> wrote:
1. Add ability to use '+' operator for dicts
Both of the ideas suffer from "+ is no longer commutative", which sort-of bothers me.
I don't find that a convincing argument, since + is not commutative for lists or tuples either. Andrii's original proposal is the most natural interpretation -- notice that if x and y are dicts: dict(x.items()) gives x dict(x.items() + y.items()) gives x + y That looks perfectly consistent to me. George's counter-proposal doesn't make sense to me at all -- it messes up the types of all the values in the dict. And it's inconsistent with the built-in behaviour of + with other types: it doesn't add lists element-by-element, so it shouldn't add dicts element-by-element either. -- ?!ng
On Tue, Mar 17, 2009 at 10:58 AM, Ka-Ping Yee <python@zesty.ca> wrote:
On Tue, 17 Mar 2009, Josiah Carlson wrote:
On Tue, Mar 17, 2009 at 9:03 AM, George Sakkis <george.sakkis@gmail.com> wrote:
On Tue, Mar 17, 2009 at 11:00 AM, Andrii V. Mishkovskyi <mishok13@gmail.com> wrote:
1. Add ability to use '+' operator for dicts
Both of the ideas suffer from "+ is no longer commutative", which sort-of bothers me.
I don't find that a convincing argument, since + is not commutative for lists or tuples either. Andrii's original proposal is the most natural interpretation -- notice that if x and y are dicts:
dict(x.items()) gives x
dict(x.items() + y.items()) gives x + y
That looks perfectly consistent to me.
George's counter-proposal doesn't make sense to me at all -- it messes up the types of all the values in the dict. And it's inconsistent with the built-in behaviour of + with other types: it doesn't add lists element-by-element, so it shouldn't add dicts element-by-element either.
Not to put words into people's mouths, but it seems like the concern was really less over the non-commutativity and move over the fact that data from the first dict gets silently clobbered by the second dict; whereas in the list, tuple, and string cases, no data is ever lost in the process. Cheers, Chris -- I have a blog: http://blog.rebertia.com
On Tue, Mar 17, 2009 at 2:14 PM, Chris Rebert <pyideas@rebertia.com> wrote:
On Tue, Mar 17, 2009 at 10:58 AM, Ka-Ping Yee <python@zesty.ca> wrote:
On Tue, 17 Mar 2009, Josiah Carlson wrote:
On Tue, Mar 17, 2009 at 9:03 AM, George Sakkis <george.sakkis@gmail.com> wrote:
On Tue, Mar 17, 2009 at 11:00 AM, Andrii V. Mishkovskyi <mishok13@gmail.com> wrote:
1. Add ability to use '+' operator for dicts
Both of the ideas suffer from "+ is no longer commutative", which sort-of bothers me.
I don't find that a convincing argument, since + is not commutative for lists or tuples either. Andrii's original proposal is the most natural interpretation -- notice that if x and y are dicts:
dict(x.items()) gives x
dict(x.items() + y.items()) gives x + y
That looks perfectly consistent to me.
George's counter-proposal doesn't make sense to me at all -- it messes up the types of all the values in the dict. And it's inconsistent with the built-in behaviour of + with other types: it doesn't add lists element-by-element, so it shouldn't add dicts element-by-element either.
Not to put words into people's mouths, but it seems like the concern was really less over the non-commutativity and move over the fact that data from the first dict gets silently clobbered by the second dict; whereas in the list, tuple, and string cases, no data is ever lost in the process.
Just to be clear, I'm between -0.5 and -1 to the whole idea; my counter-proposal was simply meant to point out the potential ambiguity in semantics and the fact that the original proposal silently loses data. George
Because there so many different ways to think about this, it's better not to guess and force the user to be explicit. On Tue, Mar 17, 2009 at 9:03 AM, George Sakkis <george.sakkis@gmail.com> wrote:
On Tue, Mar 17, 2009 at 11:00 AM, Andrii V. Mishkovskyi <mishok13@gmail.com> wrote:
1. Add ability to use '+' operator for dicts
I often wonder why list and tuple instances have '+' and '+=' operators but dicts don't? It's not that rare in my code (and code written by others, as it seems) that i have to write:
a.update(b) return a
I do understand that adding additional magic method may be inappropriate for dict, but I think it would be nice addition to a language. So, my proposal is that:
x = a + b would become equivalent to x = dict(a, **b)
a += b would become equivalent to a.update(b)
That's one way to define dict addition but it's not the only, or even, the best one. It's hard to put in words exactly why but I expect "a+b" to take into account the full state of the operands, not just a part of it. In your proposal the values of the first dict for the common keys are effectively ignored, which doesn't seem to me as a good fit for an additive operation. I would find at least as reasonable and intuitive the following definition that doesn't leak information:
def sum_dicts(*dicts): from collections import defaultdict s = defaultdict(list) for d in dicts: for k,v in d.iteritems(): s[k].append(v) return s
d1 = {'a':2,'b':5} d2 = {'a':2,'c':6,'z':3} d3 = {'b':2,'c':5} sum_dicts(d1,d2,d3) defaultdict(<type 'list'>, {'a': [2, 2], 'c': [6, 5], 'b': [5, 2], 'z': [3]})
George _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
a.update(b) return a
Why take two short, simple lines with unequivocal meaning and then abbreviate them with something mysterious (or at least something with multiple possible interpretations)? Mappings exist in many languages now. Can you point to another language that has found it worthwhile to have both an update() method and an addition operator? Also, consider that dicts are one of our most basic APIs and many other objects model that API. It behooves us to keep that API as simple and thin as possible. IMO, this change would be gratuituous. None of the code presented so far is significantly improved. Essentially, we're looking at a trivial abbreviation, not an actual offering of new capabilities. -1 all the way around. Raymond
On Tue, Mar 17, 2009 at 8:17 PM, Raymond Hettinger <python@rcn.com> wrote:
a.update(b) return a
Why take two short, simple lines with unequivocal meaning and then abbreviate them with something mysterious (or at least something with multiple possible interpretations)?
Mappings exist in many languages now. Can you point to another language that has found it worthwhile to have both an update() method and an addition operator?
Also, consider that dicts are one of our most basic APIs and many other objects model that API. It behooves us to keep that API as simple and thin as possible.
IMO, this change would be gratuituous. None of the code presented so far is significantly improved. Essentially, we're looking at a trivial abbreviation, not an actual offering of new capabilities.
Reasonable enough.
-1 all the way around.
Does that also mean -1 on list.pop() accepting slices proposal?
Raymond
_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
-- Wbr, Andrii V. Mishkovskyi. He's got a heart of a little child, and he keeps it in a jar on his desk.
George Sakkis wrote:
It's hard to put in words exactly why but I expect "a+b" to take into account the full state of the operands, not just a part of it.
I think one expects a + operator to be somehow symmetrical with respect to its operands. The lopsidedness of dict updating violates this expectation, and so is better represented by an asymmetrical syntax. -- Greg
participants (8)
-
Andrii V. Mishkovskyi -
Chris Rebert -
George Sakkis -
Greg Ewing -
Guido van Rossum -
Josiah Carlson -
Ka-Ping Yee -
Raymond Hettinger