Re: [Python-ideas] [Python-Dev] hello, new dict addition for new eve ?
Hi Julien, Don't despair! I have tried to get people to warm up to dict addition too -- in fact it was my counter-proposal at the time when we were considering adding sets to the language. I will look at your proposal, but I have a point of order first: this should be discussed on python-ideas, not on python-dev. I have added python-ideas to the thread and moved python-dev to Bcc, so followups will hopefully all go to python-ideas. --Guido On Fri, Dec 30, 2011 at 7:26 AM, julien tayon <julien@tayon.net> wrote:
Hello, Sorry to annoy the very busy core devs :) out of the blue
I quite noticed people were 1) wanting to have a new dict for Xmas 2) strongly resenting dict addition.
Even though I am not a good developper, I have come to a definition of addition that would follow algebraic rules, and not something of a dutch logic. (it is a jest, not a troll)
I propose the following code to validate my point of view regarding the dictionnatry addition as a proof of concept : https://github.com/jul/ADictAdd_iction/blob/master/test.py
It follows all my dusty math books regarding addition + it has the amability to have rules of conservation.
I pretty much see a real advantage in this behaviour in functional programming (map/reduce). (see the demonstrate.py), and it has a sense (if dict can be seen has vectors).
I have been told to be a troll, but I am pretty serious.
Since, I coded with luck, no internet, intuition, and a complete ignorance of the real meaning of the magic methods most of the time, thus the actual implementation of the addition surely needs a complete refactoring.
Sheers, Bonne fêtes Julien _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
-- --Guido van Rossum (python.org/~guido)
Hi Julien, Having now looked at your code, I realize we are not talking about the same concepts at all. What I meant is similar to set union on the keys, where if a key exists in both dicts, the value in the result is equal to one of the values in the operands (and if the value is the same for both operands, that value is also the result value). But looking at your code, you are more interested in *elementwise* addition, so that {'x': 1} + {'x': 2} results in {'x': 3}, and missing keys are considered to be zero. (For disjoint key sets, your result is the same as mine. :-) I think this would be a nice thing for some applications, but I'm doubtful that it's of enough interest to add to the standard library, let alone make it a standard feature of the built-in dict class. One problem is that the dict API is already very rich (dozens of methods and operations) Another problem is that your proposed class solves only part of the puzzle of elementwise operations -- if you have elementwise addition for dicts, why not for lists? But l1+l2 already has a meaning for list: concatenation. We've discussed this before. NumPy has got its own solution (a1+a2 does elementwise addition if a1 and a2 are NumPy arrays), but for the stdlib we've always decided that it's better to leave the choice up to the user, the app or the framework (like NumPy), instead of making a choice in the stdlib that causes backward incompatibility. (Had I known all this 22 years ago, maybe I could have chosen a different operator for list concatenation. But I don't think the issue is important enough to fix, even if we ever did a Python 4.) --Guido On Fri, Dec 30, 2011 at 9:40 AM, Guido van Rossum <guido@python.org> wrote:
Hi Julien,
Don't despair! I have tried to get people to warm up to dict addition too -- in fact it was my counter-proposal at the time when we were considering adding sets to the language. I will look at your proposal, but I have a point of order first: this should be discussed on python-ideas, not on python-dev. I have added python-ideas to the thread and moved python-dev to Bcc, so followups will hopefully all go to python-ideas.
--Guido
On Fri, Dec 30, 2011 at 7:26 AM, julien tayon <julien@tayon.net> wrote:
Hello, Sorry to annoy the very busy core devs :) out of the blue
I quite noticed people were 1) wanting to have a new dict for Xmas 2) strongly resenting dict addition.
Even though I am not a good developper, I have come to a definition of addition that would follow algebraic rules, and not something of a dutch logic. (it is a jest, not a troll)
I propose the following code to validate my point of view regarding the dictionnatry addition as a proof of concept : https://github.com/jul/ADictAdd_iction/blob/master/test.py
It follows all my dusty math books regarding addition + it has the amability to have rules of conservation.
I pretty much see a real advantage in this behaviour in functional programming (map/reduce). (see the demonstrate.py), and it has a sense (if dict can be seen has vectors).
I have been told to be a troll, but I am pretty serious.
Since, I coded with luck, no internet, intuition, and a complete ignorance of the real meaning of the magic methods most of the time, thus the actual implementation of the addition surely needs a complete refactoring.
Sheers, Bonne fêtes Julien _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
-- --Guido van Rossum (python.org/~guido <http://python.org/%7Eguido>)
-- --Guido van Rossum (python.org/~guido)
On Fri, Dec 30, 2011 at 10:02 AM, Guido van Rossum <guido@python.org> wrote:
What I meant is similar to set union on the keys, where if a key exists in both dicts, the value in the result is equal to one of the values in the operands (and if the value is the same for both operands, that value is also the result value).
+1 This is the one I was thinking of too. -eric
On Fri, Dec 30, 2011 at 9:12 AM, Eric Snow <ericsnowcurrently@gmail.com>wrote:
On Fri, Dec 30, 2011 at 10:02 AM, Guido van Rossum <guido@python.org> wrote:
What I meant is similar to set union on the keys, where if a key exists in both dicts, the value in the result is equal to one of the values in the operands (and if the value is the same for both operands, that value is also the result value).
+1
This is the one I was thinking of too.
-eric
My first thought on reading Julien's message was that {1:2} + {1:3} => {1:5} would be useful when doing reduction, There already is an operation that combines two dictionaries and does not combine values: update. Guido's version of + is a shorthand for: def __add__(self, other): result = self.copy() return result.update(other) Is saving one line of code be enough of a benefit? I think dicts are complex enough that if new functionality is added, it would be better to use named methods which can have meaningful names and are easier look up in the documentation. Furthermore, methods are more flexible as this example shows: def merge(self, other, merger=lambda x, y: x + y): """Merges another dictionary into this one, combining values.""" for k in other: if k in self: self[k] = merger(self[k], other[k]) else: self[k] = other[k] --- Bruce Follow me: http://www.twitter.com/Vroo http://www.vroospeak.com
On Fri, Dec 30, 2011 at 2:37 PM, Bruce Leban <bruce@leapyear.org> wrote:
On Fri, Dec 30, 2011 at 9:12 AM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
On Fri, Dec 30, 2011 at 10:02 AM, Guido van Rossum <guido@python.org> wrote:
What I meant is similar to set union on the keys, where if a key exists in both dicts, the value in the result is equal to one of the values in the operands (and if the value is the same for both operands, that value is also the result value).
+1
This is the one I was thinking of too.
-eric
My first thought on reading Julien's message was that {1:2} + {1:3} => {1:5} would be useful when doing reduction, There already is an operation that combines two dictionaries and does not combine values: update. Guido's version of + is a shorthand for:
def __add__(self, other): result = self.copy() return result.update(other)
The Counter class already supports this with the + operator. (It also supports element-wise max with the | operator.) Perhaps it would be useful to extend this support to dict, especially if there is a use case for element-wise string concatenation. As for a non-mutating alternative to update(): I have encountered a number of scenarios in which this is desirable. A previous suggestion [1] was to add a replace() method along these lines. But I think an operator would make it clearer which is the mutating vs. non-mutating one. Instead of +, I think << would be a better choice for this sort of modified concatenation/asymmetric union operation. For example, we can write: class Xdict(dict): def __lshift__(x, y): return Xdict(x.items() + y.items()) d = Xdict({'a': 9, 'b': 12}) d << {'a': 3, 'c': 8} # result: {'a': 3, 'b': 12, 'c': 8} d['a']==9 # True The rationale for <<: * __add__ (+) is quite common, and thus banning the addition of two dicts probably catches a lot of bugs. Moreover, when dict values are numeric, a user might expect element-wise addition (as in Counter). * __or__ (|) is classically a commutative operation, and is already overloaded by set and Counter. * Counter does not already overload <<, and thus could be extended to support the same behavior. * As far as I am aware, the only present use of << is for integer shifting, so this new idiom would not pose a conflict. Cheers, Nathan [1] http://groups.google.com/group/python-ideas/browse_thread/thread/275bbd1acfa...
Is saving one line of code be enough of a benefit?
I think dicts are complex enough that if new functionality is added, it would be better to use named methods which can have meaningful names and are easier look up in the documentation. Furthermore, methods are more flexible as this example shows:
def merge(self, other, merger=lambda x, y: x + y): """Merges another dictionary into this one, combining values.""" for k in other: if k in self: self[k] = merger(self[k], other[k]) else: self[k] = other[k]
--- Bruce Follow me: http://www.twitter.com/Vroo http://www.vroospeak.com
_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
On Fri, Dec 30, 2011 at 2:01 PM, Nathan Schneider <nathan@cmu.edu> wrote:
On Fri, Dec 30, 2011 at 2:37 PM, Bruce Leban <bruce@leapyear.org> wrote:
On Fri, Dec 30, 2011 at 9:12 AM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
On Fri, Dec 30, 2011 at 10:02 AM, Guido van Rossum <guido@python.org> wrote:
What I meant is similar to set union on the keys, where if a key exists in both dicts, the value in the result is equal to one of the values in the operands (and if the value is the same for both operands, that value is also the result value).
+1
Far more powerful is to allow an optional "collision" function that is called *only* when a key collision occurs. Special case the default behavior so that little performance hit is made if no such function is given upon initialization. Add the functionality to defaultdict and see if it gets more traction. I think you'll find a lot of code can be refactored with this simple abstraction. It becomes trivial and canonical to create a graph class for example (at least in implementation if not in interface.) To add that feature to the interface, you need to refactor the set/dict relationship so that dict inherits from set. Then you'll really cooking with gas.... Winks, mark
Not that I'm smart enough to really contribute to these discussions but wouldn't it be an idea to have dictionary addition turn the values from d2 into d1 so that d1 = {1:1,2:2,3:3} d2 = {1:2,2:2,4:4} d3 = d1 + d2 print d3 # {1:2,2:2,3:3,4:4} # but d4 = d2 + d1 print d4 # {1:1,2:2,3:3,4:4} Meaning that replacement is conditional on which side the other dictionary is on with the + operator. This might be harder to understand. So it's not a great idea... Still throwing it in the ring is the only way to learn Sent from my iPad On 30 Dec 2011, at 17:12, Eric Snow <ericsnowcurrently@gmail.com> wrote:
On Fri, Dec 30, 2011 at 10:02 AM, Guido van Rossum <guido@python.org> wrote:
What I meant is similar to set union on the keys, where if a key exists in both dicts, the value in the result is equal to one of the values in the operands (and if the value is the same for both operands, that value is also the result value).
+1
This is the one I was thinking of too.
-eric _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
On 12/30/2011 12:02 PM, Guido van Rossum wrote:
But looking at your code, you are more interested in *elementwise* addition, so that {'x': 1} + {'x': 2} results in {'x': 3}, and missing keys are considered to be zero.
This is part of what collections.Counter does.
(Had I known all this 22 years ago, maybe I could have chosen a different operator for list concatenation.
In base 1 notation, count addition amounts to sequence concatenation. So I think you intuitively made the right choice. -- Terry Jan Reedy
On Fri, Dec 30, 2011 at 7:26 AM, julien tayon <julien@tayon.net <mailto:julien@tayon.net>> wrote:
I quite noticed people were 1) wanting to have a new dict for Xmas
Someone has proposed a modified version of the current dict type. The questions are whether a) it will behave correctly in all situations; b) give better overall performance; and c) be as easily maintainable.
2) strongly resenting dict addition.
This strikes me as a nonsensical statement. As noted below, we already have a type of 'dict addition' -- and even 'dict subtraction'. Counterfactual negative statements like this lead to the impression that you are trolling, even if you do not intend to.
I propose the following code to validate my point of view regarding the dictionnatry addition as a proof of concept : https://github.com/jul/ADictAdd_iction/blob/master/test.py
This link any gives the tests, not the accu_dict itself. So there is no way to know from the above exactly what you are proposing. From what Guido wrote in his next response, I am guessing that it is something like collections.Counter, which is an implementation of the multiset idea.
c = Counter(a=3, b=1) d = Counter(a=1, b=2) c + d # add two counters together: c[x] + d[x] Counter({'a': 4, 'b': 3})
c - d # subtract (keeping only positive counts) Counter({'a': 2})
Tagging numbers by keys is a generalization of tagging them by position (index). So this is generalized vector arithmetic and that is what your tests test. ------------------------ You have mixed up some of the terminology. def test_add_permut(self): """( a + b ) + c = a + ( b + c )""" This is the associative law, not 'permutative'. def test_permut_scal_mul(self): """ ( an_int * other_scalar ) * a = an_int ( other_scalar * a )""" Again, associativity (of scalar multiplication). --- def test_other_mul_scal_commut(self): """ ( an_int + other_scalar ) a = an_int * a + other_scalar * a """ This is a distributive law. def test_scal_lin_combo(self): """ an_int ( a + b) = an_int *a + an_int * b """ So is this. --- def test_permutation(self): """a + b = b + a """ This is commutivity, which you had right earlier in the file: def test_commutativity(self): """ a + b = b + a """ --- def test_neutral(self): """ a + neutral = a """ Not exactly incorrect, but this is a test of the additive identify, which is usually called 'zero' rather than 'neutral'. def test_neutral_mul_scal(self): """ 1 * a = a """ while this tests the scalar multiplicative identify -- Terry Jan Reedy
On 30 December 2011 20:29, Terry Reedy <tjreedy@udel.edu> wrote: [...]
You have mixed up some of the terminology. [...] --- def test_permutation(self): """a + b = b + a """
This is commutivity, which you had right earlier in the file:
You mean commutativity :) [...]
def test_neutral(self): """ a + neutral = a """
Not exactly incorrect, but this is a test of the additive identify, which is usually called 'zero' rather than 'neutral'.
You mean additive identity (by the way, I think Julien used the word 'neutral' because in French we call it 'élément neutre')
def test_neutral_mul_scal(self): """ 1 * a = a """
while this tests the scalar multiplicative identify
You mean multiplicative identity :) Nit-picklingly yours, -- Arnaud
participants (8)
-
Arnaud Delobelle -
Bruce Leban -
Eric Snow -
Guido van Rossum -
Jakob Bowyer -
Mark Janssen -
Nathan Schneider -
Terry Reedy