From what I read, it seems the biggest concern is: which value to pick up if both dicts have the same key. a = {'x':1} b = {'x':2} c = a | b
Hello I know this has been mangled thousand times, but let's do it once again. Why does Python not have a simple dict joining operator? print(c['x']) # 1 or 2? My proposal is: the value should be derermined as the result of the operation 'or'. The 'or' operator returns the first operand that evaluates to boolean True, or the last operand if all are False. So, provided we have 2 dicts 'a' and 'b' and c = a | b 1. If a key is not present in 'a' nor 'b', then it is not present in c. 2. If a key is present in 'a' and not in 'b', then c[k] = a[k]. 3. If a key is present in 'b' and not in 'a', then c[k] = b[k]. 4. If a key is present both in 'a' and 'b', then c[k] = a[k] or b[k]. We could also naturally define dict intersection operator using the operator 'and' to pick values. The key would have to be present in both operands. Forgive me I wasn't able to read all previous discussions. Was this proposed before? Thanks haael
On Fri, Feb 21, 2014 at 8:40 PM, <haael@interia.pl> wrote:
I know this has been mangled thousand times, but let's do it once again.
Why does Python not have a simple dict joining operator?
From what I read, it seems the biggest concern is: which value to pick up if both dicts have the same key. a = {'x':1} b = {'x':2} c = a | b print(c['x']) # 1 or 2?
If you can pick one of the dicts to "win", then you can use this notation: c = dict(a, **b) Anything in b will override a. It's short and a single expression (unlike "c = a.copy(); c.update(b)"). It's not perfectly clear what's happening, though, so it may warrant a comment. ChrisA
...but that doesn't work if you have non-string keys. http://stackoverflow.com/a/39858/2097780 On Fri, Feb 21, 2014 at 4:15 AM, Chris Angelico <rosuav@gmail.com> wrote:
On Fri, Feb 21, 2014 at 8:40 PM, <haael@interia.pl> wrote:
I know this has been mangled thousand times, but let's do it once again.
Why does Python not have a simple dict joining operator?
From what I read, it seems the biggest concern is: which value to pick up if both dicts have the same key. a = {'x':1} b = {'x':2} c = a | b print(c['x']) # 1 or 2?
If you can pick one of the dicts to "win", then you can use this notation:
c = dict(a, **b)
Anything in b will override a. It's short and a single expression (unlike "c = a.copy(); c.update(b)"). It's not perfectly clear what's happening, though, so it may warrant a comment.
ChrisA _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated."
On 21 February 2014 09:40, <haael@interia.pl> wrote:
From what I read, it seems the biggest concern is: which value to pick up if both dicts have the same key. a = {'x':1} b = {'x':2} c = a | b print(c['x']) # 1 or 2?
My proposal is: the value should be derermined as the result of the operation 'or'. The 'or' operator returns the first operand that evaluates to boolean True, or the last operand if all are False.
So, provided we have 2 dicts 'a' and 'b' and c = a | b
1. If a key is not present in 'a' nor 'b', then it is not present in c. 2. If a key is present in 'a' and not in 'b', then c[k] = a[k]. 3. If a key is present in 'b' and not in 'a', then c[k] = b[k]. 4. If a key is present both in 'a' and 'b', then c[k] = a[k] or b[k].
This seems to me to be exactly the same as dict.update(): a = {'x': 1} b = {'x': 2} c = b.copy() c.update(a) I'd be very reluctant to overload the bitwise-or operator to mean 'dictionary merging'. It's just not totally obvious to me that this is what would happen.
On 2014-02-21 11:17, Cory Benfield wrote:
On 21 February 2014 09:40, <haael@interia.pl> wrote:
From what I read, it seems the biggest concern is: which value to pick up if both dicts have the same key. a = {'x':1} b = {'x':2} c = a | b print(c['x']) # 1 or 2?
My proposal is: the value should be derermined as the result of the operation 'or'. The 'or' operator returns the first operand that evaluates to boolean True, or the last operand if all are False.
So, provided we have 2 dicts 'a' and 'b' and c = a | b
1. If a key is not present in 'a' nor 'b', then it is not present in c. 2. If a key is present in 'a' and not in 'b', then c[k] = a[k]. 3. If a key is present in 'b' and not in 'a', then c[k] = b[k]. 4. If a key is present both in 'a' and 'b', then c[k] = a[k] or b[k].
This seems to me to be exactly the same as dict.update():
a = {'x': 1} b = {'x': 2}
c = b.copy() c.update(a)
It seems that everybody misses the part of the OP where he states that conflict resolution shouldn't happen via "one dict wins" but rather with the "or"-operator.
I'd be very reluctant to overload the bitwise-or operator to mean 'dictionary merging'. It's just not totally obvious to me that this is what would happen. _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Fri, Feb 21, 2014 at 05:05:58PM +0100, Markus Unterwaditzer wrote: [...]
It seems that everybody misses the part of the OP where he states that conflict resolution shouldn't happen via "one dict wins" but rather with the "or"-operator.
I certainly did :-( So we have *at least* four different ways to merge dictionaries a and b: # 1: a wins c = b.copy() c.update(a) # 2: b wins c = a.copy() c.update(b) # 3: choose a winner according to the `or` operator c = a.copy() for key, value in b.items(): if key in c: c[key] = c[key] or value else: c[key] = value # 4: keep both, in a list of 1 or 2 items c = {key:[value] for key, value in a.items()} for key, value in b.items(): if key in c and value != c[key][0]: c[key].append(value) else: c[key] = [value] The first three are special cases of a more general case, where you have a "decision function" that takes two values (one from dict a and the other from dict b) and decides which one to keep. Case 1 ("a always wins") would use `lambda x,y: x`, case 2 ("b wins") would use `lambda x,y: y` and case 3 would use operator.or_. The question is, why should any one of these be picked out as so obviously more useful than the others as to deserve being a dict method or operator support? -- Steven
Which opens up the same issue I mentioned earlier: non-string keys don't work. On Fri, Feb 21, 2014 at 6:50 PM, Jan Kaliszewski <zuo@chopin.edu.pl> wrote:
21.02.2014 23:57, Steven D'Aprano wrote:
# 1: a wins
c = b.copy() c.update(a)
Sometimes I do:
c = dict(b, **a)
Cheers. *j
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated."
@Steven D'Aprano
So we have *at least* four different ways to merge dictionaries a and b:
# 1: a wins c = b.copy() c.update(a)
# 2: b wins c = a.copy() c.update(b)
# 3: choose a winner according to the `or` operator c = a.copy() for key, value in b.items(): if key in c: c[key] = c[key] or value else: c[key] = value
# 4: keep both, in a list of 1 or 2 items c = {key:[value] for key, value in a.items()} for key, value in b.items(): if key in c and value != c[key][0]: c[key].append(value) else: c[key] = [value]
The first three are special cases of a more general case, where you have a "decision function" that takes two values (one from dict a and the other from dict b) and decides which one to keep. Case 1 ("a always wins") would use `lambda x,y: x`, case 2 ("b wins") would use `lambda x,y: y` and case 3 would use operator.or_.
The question is, why should any one of these be picked out as so obviously more useful than the others as to deserve being a dict method or operator support?
Steven
All solutions provided by you are not one-liners. Every requires at least 2 lines of code and is an imperative-style code block, instead of a simple expression. I would really like to have a simple dict joining _expression_ that can be inserted everywhere I just need. fun(dict_arg=(dict_a | dict_b)) fun(**(dict_a | dict_b)) I personally have no problem with any of your code samples being promoted to an operator. My proposal is philosophically exactly the same as dropping the "print" _statement_ and replacing it with "print" _function_. It just merges more nicely with the rest of Python code. Really, I would even be happy if we had at least a dict _method_ that returns the updated dict: {'a':1}.update({'b':2}) # returns {'a':1, 'b':2} @Mathias Panzenböck
I never had a case where this kind of conflict resolution made sense. Can you show us an example?
Consider this example once again: fun(**(dict_a | dict_b)) Function default parameters are often null-like values. I don't say that they always are, but if is common in the programming practice. The null values usually evaluate to boolean False in Python when converted. Now consider the following: Let's have a dict with some function's default parameters: {'a':None, 'b':[]} Now let's have two separate code blocks that fill those arguments with some computed values. def funA(p): p['a'] = 1 return p def funB(p): p[b] = [1, 2, 3] return p Again, this pattern is not uncommon in the programming practice. We often have some code blocks that try to fill as much parameters as possible, but not every of them. Finally, we want to merge the dicts returned by these functions and provide them to the function of our interest. Basically, we want something like that: dict_default = {'a':None, 'b':[]} dict_a = funA(dict_default.copy()) dict_b = funB(dict_default.copy()) dict_param = merge_using_or_resolution(dict_a, dict_b) fun(**dict_param) Quite a lot of code, as you see. It also involves a lot of copying. The function 'merge_using_or_resolution' also needs at least one dict copy. If we had a dict joining operator, we could simply write: dict_default = {'a':None, 'b':[]} fun(**(funA(dict_default) | funB(dict_default)) No copies, no temporary variables, the functions funA and funB would also become one-liners, no namespace and memory pollution. Also, this code could be better optimized, unlike the previous example. As you see, there are many benefits of having dict-joining operator. I want to stress it once again. My examples are quite abstract, but the code patterns are actually quite common. Thanks haael
On Sun, Feb 23, 2014 at 02:31:27PM +0100, haael@interia.pl wrote:
@Steven D'Aprano
So we have *at least* four different ways to merge dictionaries a and b:
[...]
The question is, why should any one of these be picked out as so obviously more useful than the others as to deserve being a dict method or operator support?
Please note the question. That is a critical question.
I would really like to have a simple dict joining _expression_ that can be inserted everywhere I just need.
But which one? There are at least four "simple dict joining" operations. Which one should be turned into a method, and which ones should be left out? It's easy to say that you want a dict operator | to merge dictionaries, but you still have to explain why one merge function should be promoted to an operator, and the others three (or more) merge functions miss out. Then, when people complain that *their* choice got left out, you can explain why their use-case is not important enough to be an operator, but yours is. In the meantime, you can add a two or three line function to your module, and use that. An easy solution to a simple problem. -- Steven
On Sun, Feb 23, 2014 at 02:31:27PM +0100, haael@interia.pl wrote:
@Mathias Panzenböck
I never had a case where this kind of conflict resolution made sense. Can you show us an example? [...] Now consider the following:
Let's have a dict with some function's default parameters: {'a':None, 'b':[]}
Seems a bit contrived, but okay.
Now let's have two separate code blocks that fill those arguments with some computed values.
def funA(p): p['a'] = 1 return p
def funB(p): p[b] = [1, 2, 3] return p
I notice that both of these mutate the dictionary in place. This is important later on.
Again, this pattern is not uncommon in the programming practice. We often have some code blocks that try to fill as much parameters as possible, but not every of them.
I can't say that I've ever done this, but let's continue.
Finally, we want to merge the dicts returned by these functions and provide them to the function of our interest.
Basically, we want something like that:
dict_default = {'a':None, 'b':[]} dict_a = funA(dict_default.copy()) dict_b = funB(dict_default.copy()) dict_param = merge_using_or_resolution(dict_a, dict_b) fun(**dict_param)
I see that you are making dict_a and dict_b mutated copies. I don't understand why you don't just work with a single dict, instead of making copies all over the place. And since both funA and funB return the dict, you can chain them. This is so such simpler than the way you wrote it, and avoids all the unnecessary copying: dict_param = {'a':None, 'b':[]} fun(**(funA(funB(dict_param)))) Even if you decide that you want to keep the defaults untouched (in case you need them later), that makes only a single copy: dict_defaults = {'a':None, 'b':[]} fun(**(funA(funB(dict_defaults.copy())))) -- Steven
On Mon, Feb 24, 2014 at 1:20 AM, Steven D'Aprano <steve@pearwood.info> wrote:
Let's have a dict with some function's default parameters: {'a':None, 'b':[]}
Seems a bit contrived, but okay.
I hesitate to overly optimize contrived examples. Usually results in missing one critical part of what we weren't told and being useless. :| ChrisA
On Mon, Feb 24, 2014 at 12:31 AM, <haael@interia.pl> wrote:
dict_default = {'a':None, 'b':[]} fun(**(funA(dict_default) | funB(dict_default))
All you need is a shorter name for your function and the job's done. Since it'll be local to your module anyway, it doesn't need the long and verbose name. Here's Steven's #3, implemented as a function: def merge(a, b): "Merge two dicts using the 'or' operator c = a.copy() for key, value in b.items(): if key in c: c[key] = c[key] or value else: c[key] = value Or you could make it shorter thus: def merge(a, b): "Merge two dicts using the 'or' operator c = a.copy() for key, value in b.items(): c[key] = c.get(key) or value Then your code is: dict_default = {'a':None, 'b':[]} fun(**merge(funA(dict_default), funB(dict_default))) That's it! ChrisA
On 23/02/2014 13:31, haael@interia.pl wrote:
@Steven D'Aprano
So we have *at least* four different ways to merge dictionaries a and b:
# 1: a wins c = b.copy() c.update(a)
# 2: b wins c = a.copy() c.update(b)
# 3: choose a winner according to the `or` operator c = a.copy() for key, value in b.items(): if key in c: c[key] = c[key] or value else: c[key] = value
# 4: keep both, in a list of 1 or 2 items c = {key:[value] for key, value in a.items()} for key, value in b.items(): if key in c and value != c[key][0]: c[key].append(value) else: c[key] = [value]
The first three are special cases of a more general case, where you have a "decision function" that takes two values (one from dict a and the other from dict b) and decides which one to keep. Case 1 ("a always wins") would use `lambda x,y: x`, case 2 ("b wins") would use `lambda x,y: y` and case 3 would use operator.or_.
The question is, why should any one of these be picked out as so obviously more useful than the others as to deserve being a dict method or operator support?
Steven
All solutions provided by you are not one-liners. Every requires at least 2 lines of code and is an imperative-style code block, instead of a simple expression.
Excellent as it makes the code easier to read. Let's leave things like that.
I would really like to have a simple dict joining _expression_ that can be inserted everywhere I just need.
I wouldn't. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com
On 02/21/2014 10:40 AM, haael@interia.pl wrote:
Hello
I know this has been mangled thousand times, but let's do it once again.
Why does Python not have a simple dict joining operator?
From what I read, it seems the biggest concern is: which value to pick up if both dicts have the same key. a = {'x':1} b = {'x':2} c = a | b print(c['x']) # 1 or 2?
My proposal is: the value should be derermined as the result of the operation 'or'. The 'or' operator returns the first operand that evaluates to boolean True, or the last operand if all are False.
So, provided we have 2 dicts 'a' and 'b' and c = a | b
1. If a key is not present in 'a' nor 'b', then it is not present in c. 2. If a key is present in 'a' and not in 'b', then c[k] = a[k]. 3. If a key is present in 'b' and not in 'a', then c[k] = b[k]. 4. If a key is present both in 'a' and 'b', then c[k] = a[k] or b[k].
We could also naturally define dict intersection operator using the operator 'and' to pick values. The key would have to be present in both operands.
Forgive me I wasn't able to read all previous discussions. Was this proposed before?
I don't see how this significantly differs from dict update, except that for you the first one to speak should win, instead of the opposite; so that you have to reverse the operands. d
Am 2014-02-21 10:40, schrieb haael@interia.pl:
Hello
I know this has been mangled thousand times, but let's do it once again.
Why does Python not have a simple dict joining operator?
From what I read, it seems the biggest concern is: which value to pick up if both dicts have the same key. a = {'x':1} b = {'x':2} c = a | b print(c['x']) # 1 or 2?
My proposal is: the value should be derermined as the result of the operation 'or'. The 'or' operator returns the first operand that evaluates to boolean True, or the last operand if all are False.
So, provided we have 2 dicts 'a' and 'b' and c = a | b
1. If a key is not present in 'a' nor 'b', then it is not present in c. 2. If a key is present in 'a' and not in 'b', then c[k] = a[k]. 3. If a key is present in 'b' and not in 'a', then c[k] = b[k]. 4. If a key is present both in 'a' and 'b', then c[k] = a[k] or b[k].
I never had a case where this kind of conflict resolution made sense. Can you show us an example?
We could also naturally define dict intersection operator using the operator 'and' to pick values. The key would have to be present in both operands.
Forgive me I wasn't able to read all previous discussions. Was this proposed before?
Thanks haael
participants (10)
-
Chris Angelico -
Cory Benfield -
haael@interia.pl -
Jan Kaliszewski -
Mark Lawrence -
Markus Unterwaditzer -
Mathias Panzenböck -
Ryan Gonzalez -
spir -
Steven D'Aprano