On Sat, Dec 31, 2011 at 4:55 PM, Terry Reedy wrote:

On 12/31/2011 11:07 AM, julien tayon wrote:

...

The question is a dict the same as a vector or as a set ?

A dict is a mapping, which is to say, a functional set of key,value pairs. 'Functional' means that the keys are unique. This is a key property (pun intended). A vector can be regarded as a functional set of count,value pairs. Some languages use dicts for vectors. Even in Python, people use dicts for sparse arrays of various dimensions, with unspecified pairs assumed to have a 0 value.

So it makes the most sense to me for d1 + d2 to be interpreted as set union, with some resolution of key conflicts to restore the functional property. Your vector interpretation of adding values only works for values that can be added.

It occurs to me that any argument for a vector interpretation of dicts, with element-wise operations, should apply equally well to lists and tuples, which are perhaps even more natural as representations for vectors. But of course, the + operator for sequences is interpreted as concatenation, not element-wise addition; it would be strange if the reverse were true for dicts. The necessary resolution of key conflicts is what makes "+" feel less natural to me for dicts than it does for sequences, where the order of the operands is transparently reflected in the output. For dicts, in contrast, we are talking about a "lossy" union/combination where what is lost depends on the order of the operands—this is less transparent in the output. However it is spelled, though, I would be thrilled if such an operation were built in. :) Nathan

...

since sets are using logical operators for operations that are roughly sets operations why not use &  ^ | for sets operation on dict ? it would be pretty consistent with sets operations.

I agree, except that dicts are not just sets of pairs, but functional sets. If one wants to view a dict simply as a set of keys, values, or pairs, ignoring the functional property for the pairs, that is what the .keys(), .values(), and .items() set view methods are for. All the set operations are available on the resulting set view objects.

-- Terry Jan Reedy

_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas

On Sun, Jan 1, 2012 at 3:24 AM, Terry Reedy wrote:

You mean like dict.update?

Nah nah, that's "+= "; "+" is dict(d1, **d2). -- Devin On Sun, Jan 1, 2012 at 3:24 AM, Terry Reedy wrote:

On 12/31/2011 5:57 PM, Nathan Schneider wrote:

...

The necessary resolution of key conflicts is what makes "+" feel less natural to me for dicts than it does for sequences, where the order of the operands is transparently reflected in the output. For dicts, in contrast, we are talking about a "lossy" union/combination where what is lost depends on the order of the operands—this is less transparent in the output. However it is spelled, though, I would be thrilled if such an operation were built in. :)

You mean like dict.update?

-- Terry Jan Reedy

_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas

Here are some "utility" functions I had to write in my current project: # add entries from d2 to d1 # supports only dict and numeric types for now # recursive, applies function to all subdicts def addDictEntries(d1, d2) # take the max of entries from d2 to d1 # supports only dict and numeric types for now # recursive, applies function to all subdicts def maxDictEntries(d1, d2): # take the min of entries from d2 to d1 # supports only dict and numeric types for now # recursive, applies function to all subdicts def minDictEntries(d1, d2) # merges entries from d2 to d1 like update does # if an entry already exists and the values differ then throws an exception # recursive, applies function to all subdicts def mergeDictEntriesSafe(d1, d2) # traverses through the dict and converts all defaultdicts to dicts # this is done so that missing elements will throw an exception rather than creating a default entry # also, defaultdicts aren't picklable so this is required in order to serialize a dict def unDefaultDict(d) # same as unDefaultDict but isn't recursive. Only applies to top level keys def unDefaultDictShallow(d) The usage for all this varies but I found that combining / adding dicts together becomes very prevalent when you split datasets up for multihreading/multiprocessing purposes and need to recombine it later. so I propose that a "combine" function is added to the dict class and all dict subclasses. The function would have the following arguments: def combine(self, d2, conflictFunc=None): If combineFunc is a function, then that function is called and the result is used. val = conflictFunc(k1,k2); If combineFunc is a string, then self's key value calls conflictFunc and the new value is the return value def combine(self, d2, combineFunc=None): if combineFunc == None: for k,i in d2.items(): if(not k in self): self[k] = copy.deepcopy(i); else: if(hasattr(i,'keys')): self[k].combine(i); else: if self[k] != i: raise someUsefulException(); return; if (type(combineFunc) == str): for k,i in d2.items(): if(not k in self): self[k] = copy.deepcopy(i); else: if(hasattr(i,'keys')): self[k].combine(i,combineFunc); else: self[k] = getattr(self[k],combineFunc)(i); else: for k,i in d2.items(): if(not k in self): self[k] = copy.deepcopy(i); else: if(hasattr(i,'keys')): self[k].combine(i,combineFunc); else: self[k] = combineFunc(self[k],i); examples: d.combine(d2); # update with d2 but throw an exception if any keys have different values d.combine(d2, myCustomFunc); # if conflict, then result is = myCustomFunc(v1,v2) which would be defined elsewhere d.combine(d2, '__add__'); # if conflict, then result is = v1 + v2

...

print(d1) {'a': 'b', 'c': {'g': 5}} print(d2) {'a': 'ecause', 'c': {'g': 10}} d1.combine(d2,'__add__'); print(d1) {'a': 'because', 'c': {'g': 15}}

I think doing so would be more powerful and less ambiguous than trying to give dict a '+' A second function I suggest adding is asDict(). This would return a dictionary with all sub dictionaries-types converted to the dict type. I.E. all defaultdicts and counters become just plain dicts. This would be the same as my unDefaultDict function. It could take an optional argument to do the conversion only to the shallow keys def asDict(self, shallow=False)

This is slightly tangential, but I've always wondered... Why aren't set operations implemented on dicts? It is fairly natural to view a dictionary as a set of (key, value) pairs. Things like subset/superset checking (with concomitant operator support) make sense. I have written stuff like set(dict1.items()) < set(dict2.items()) many times. I don't even know what rich comparison operators on dictionaries do now, it isn't intuitive at all. Nathan

On Tue, Jan 3, 2012 at 12:10 PM, Robert Kern wrote:

On 1/3/12 4:59 PM, Nathan Rice wrote:

...

This is slightly tangential, but I've always wondered... Why aren't set operations implemented on dicts?  It is fairly natural to view a dictionary as a set of (key, value) pairs.  Things like subset/superset checking (with concomitant operator support) make sense.  I have written stuff like set(dict1.items())< set(dict2.items()) many times.

The values are unrestricted Python objects. They do not have to be hashable or sortable. The set operations you describe would have to be require one or both (or else do something algorithmically horrendous).

I haven't had any problems with using set(somedict.items()). I will admit that I primarily do this in simple contexts. This brings me to another curiosity... Why do mutable items not implement hash using id()?

Further, you cannot treat dicts as sets of (key, value) pairs because dicts have unique keys, not unique (key, value) pairs.

I'm confused. Because keys are unique, (key, value) pairs are unique as well. I realize the values are not unique but the tuple is guaranteed to be.

...

I don't even know what rich comparison operators on dictionaries do now, it isn't intuitive at all.

The rich comparison operators are only defined for == and !=. In Python 2.x, there is a legacy implementation of __cmp__ that does something more complicated. I recommend ignoring it.

Recommendation definitely noted :D Nathan

On 3 January 2012 18:45, Robert Kern wrote:

[~] |7> a = {'one': 1}

[~] |8> b = {'one': '1'}

[~] |9> set(a.items()) | set(b.items()) set([('one', '1'), ('one', 1)])

[~] |10> dict(set(a.items()) | set(b.items())) {'one': 1}

I don't know what you've done here:

...

...

a = {'one': 1} b = {'one': 1} set(a.items()) | set(b.items()) {('one', 1)} dict(set(a.items()) | set(b.items())) {'one': 1}

( However, your point remains:

...

...

a = {'number': 1} b = {'number': 2} set(a.items()) | set(b.items()) {('number', 1), ('number', 2)} dict(set(a.items()) | set(b.items())) {'number': 2}

)

On Tue, Jan 3, 2012 at 1:45 PM, Robert Kern wrote:

On 1/3/12 5:39 PM, Nathan Rice wrote:

...

I haven't had any problems with using set(somedict.items()).  I will admit that I primarily do this in simple contexts.

This brings me to another curiosity... Why do mutable items not implement hash using id()?

Usually because they do not implement __eq__ using id(). The invariant that needs to be maintained is that if two objects compare equal, then they need to hash equal. Many mutable objects compare by value, and thus two non-identical objects can compare equal.

Ok. I missed that invariant in the data model (and a couple of other places, apparently). Thanks.

...

...

Further, you cannot treat dicts as sets of (key, value) pairs because dicts have unique keys, not unique (key, value) pairs.

I'm confused.  Because keys are unique, (key, value) pairs are unique as well.  I realize the values are not unique but the tuple is guaranteed to be.

Yes, the dict.items() would be a valid, unique set, but set operations on those sets may not give valid dicts because the same key could point to different values in the two dict operands. For example:

[~] |7> a = {'one': 1}

[~] |8> b = {'one': '1'}

[~] |9> set(a.items()) | set(b.items()) set([('one', '1'), ('one', 1)])

[~] |10> dict(set(a.items()) | set(b.items())) {'one': 1}

You've lost a value there.

I have been following the earlier discussion of + in the context of dictionaries, and I agree that many operators do not cleanly map over. It seems to me that the comparison operators are well behaved in this regard though. As an additional aside, perhaps the return type of elements in the items call could be made into a nice "Item" namedtuple. I always unpack my items into (k, v) or something similar when possible, but some people aren't as considerate. Having item.key and item.value available (and used in examples) encourages nice code; even if someone chose a horrible name for their items() unpacking, it is easier to grok "badname.key ... badname.value" than "badname[0] ... badname[1]". Nathan

The values are unrestricted Python objects. They do not have to be hashable or sortable. The set operations you describe would have to be require one or both (or else do something algorithmically horrendous).

He only describes <, which can be implemented in linear time as: def __lt__(self, d2): if not isinstance(d2, dict): return NotImplemented return all(key in d2 and d2[key] == value for key, value in self.items()) Which set operations are you thinking of? -- Devin On Tue, Jan 3, 2012 at 12:10 PM, Robert Kern wrote:

On 1/3/12 4:59 PM, Nathan Rice wrote:

...

This is slightly tangential, but I've always wondered... Why aren't set operations implemented on dicts?  It is fairly natural to view a dictionary as a set of (key, value) pairs.  Things like subset/superset checking (with concomitant operator support) make sense.  I have written stuff like set(dict1.items())< set(dict2.items()) many times.

The values are unrestricted Python objects. They do not have to be hashable or sortable. The set operations you describe would have to be require one or both (or else do something algorithmically horrendous).

Further, you cannot treat dicts as sets of (key, value) pairs because dicts have unique keys, not unique (key, value) pairs.

...

I don't even know what rich comparison operators on dictionaries do now, it isn't intuitive at all.

The rich comparison operators are only defined for == and !=. In Python 2.x, there is a legacy implementation of __cmp__ that does something more complicated. I recommend ignoring it.

-- Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma  that is made terrible by our own mad attempt to interpret it as though it had  an underlying truth."  -- Umberto Eco

_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas

On 1/3/2012 11:59 AM, Nathan Rice wrote:

This is slightly tangential, but I've always wondered... Why aren't set operations implemented on dicts? It is fairly natural to view a dictionary as a set of (key, value) pairs. Things like subset/superset checking (with concomitant operator support) make sense. I have written stuff like set(dict1.items())< set(dict2.items()) many times.

As a said earlier in this thread, being able to view dicts as sets is such a good idea that it was already added a few years ago.

...

...

{1:1}.items() < {1:1, 2:2}.items() True

I don't even know what rich comparison operators on dictionaries do now, it isn't intuitive at all.

TypeError: unorderable types: dict() < dict() -- Terry Jan Reedy

2012/1/3 Nathan Rice :

...

As a said earlier in this thread, being able to view dicts as sets is such a good idea that it was already added a few years ago.

...

...

...

{1:1}.items() < {1:1, 2:2}.items() True

Making me think of funny stuffs :

...

...

dict( a = .1 * .1 ) > dict( a = .01 ) True ( value compared, floats are behaving as they always do)

...

...

dict( a = lambda x : x ) == dict( a = lambda x : x ) False ( value compared, and lambda are id compared I guess)

And since I always mixed up truth and lies :

...

...

False , True = True, False == True and "to be" or not "to be" if "logic" is "insane" else 42 -- Jul