[Python-ideas] [Python-Dev] hello, new dict addition for new eve ?

[James Hutchison]

Here are some "utility" functions I had to write in my current project:

# add entries from d2 to d1
# supports only dict and numeric types for now
# recursive, applies function to all subdicts
def addDictEntries(d1, d2)

# take the max of entries from d2 to d1
# supports only dict and numeric types for now
# recursive, applies function to all subdicts
def maxDictEntries(d1, d2):

# take the min of entries from d2 to d1
# supports only dict and numeric types for now
# recursive, applies function to all subdicts
def minDictEntries(d1, d2)

# merges entries from d2 to d1 like update does
# if an entry already exists and the values differ then throws an exception
# recursive, applies function to all subdicts
def mergeDictEntriesSafe(d1, d2)

# traverses through the dict and converts all defaultdicts to dicts
# this is done so that missing elements will throw an exception rather
than creating a default entry
# also, defaultdicts aren't picklable so this is required in order to
serialize a dict
def unDefaultDict(d)

# same as unDefaultDict but isn't recursive. Only applies to top level keys
def unDefaultDictShallow(d)

The usage for all this varies but I found that combining / adding
dicts together becomes very prevalent when you split datasets up for
multihreading/multiprocessing purposes and need to recombine it later.

so I propose that a "combine" function is added to the dict class and
all dict subclasses. The function would have the following arguments:

def combine(self, d2, conflictFunc=None):

If combineFunc is a function, then that function is called and the
result is used. val = conflictFunc(k1,k2);
If combineFunc is a string, then self's key value calls conflictFunc
and the new value is the return value

    def combine(self, d2, combineFunc=None):
        if combineFunc == None:
            for k,i in d2.items():
                if(not k in self):
                    self[k] = copy.deepcopy(i);
                else:
                    if(hasattr(i,'keys')):
                        self[k].combine(i);
                    else:
                        if self[k] != i:
                            raise someUsefulException();
            return;
        if (type(combineFunc) == str):
            for k,i in d2.items():
                if(not k in self):
                    self[k] = copy.deepcopy(i);
                else:
                    if(hasattr(i,'keys')):
                        self[k].combine(i,combineFunc);
                    else:
                        self[k] = getattr(self[k],combineFunc)(i);
        else:
            for k,i in d2.items():
                if(not k in self):
                    self[k] = copy.deepcopy(i);
                else:
                    if(hasattr(i,'keys')):
                        self[k].combine(i,combineFunc);
                    else:
                        self[k] = combineFunc(self[k],i);

examples:

d.combine(d2); # update with d2 but throw an exception if any keys
have different values
d.combine(d2, myCustomFunc); # if conflict, then result is =
myCustomFunc(v1,v2) which would be defined elsewhere
d.combine(d2, '__add__'); # if conflict, then result is = v1 + v2

>>print(d1)
{'a': 'b', 'c': {'g': 5}}
>>print(d2)
{'a': 'ecause', 'c': {'g': 10}}
>>d1.combine(d2,'__add__');
>>print(d1)
{'a': 'because', 'c': {'g': 15}}

I think doing so would be more powerful and less ambiguous than trying
to give dict a '+'

A second function I suggest adding is asDict(). This would return a
dictionary with all sub dictionaries-types converted to the dict type.
I.E. all defaultdicts and counters become just plain dicts. This would
be the same as my unDefaultDict function. It could take an optional
argument to do the conversion only to the shallow keys

def asDict(self, shallow=False)