defaultdicts pickling

[Gabriel Genellina]

En Sat, 14 Jul 2007 13:03:01 -0300, Yoav Goldberg  
<yoav.goldberg at gmail.com> escribió:

> I need to have a dictionary of dictionaries of numbers, and I would like  
> the
> dictionaries to be defaultdicts, because it makes the code much nicer
> (I want to be able to do:  d['foo']['bar']+=1 ).
>
> So naturally, I used:
>
> d = defaultdict(lambda :defaultdict(int))
>
> It works great, but now I can not pickle it.
>
> I could ofcourse used a real function and not a lambda, but this would  
> make
> things (a) somewhat slower and (b) a bit ugly.

(a) AFAIK, lambdas are true functions, just without name, nor docstrings,  
nor decorator support, and they only allow expressions. But the execution  
time should be similar:

 from collections import defaultdict

def test_lambda():
     d = defaultdict(lambda: defaultdict(int))
     for i in range(100):
         for j in range(100):
             d[i%7][j%7] += 1

def test_func():
     def func(): return defaultdict(int)
     d = defaultdict(func)
     for i in range(100):
         for j in range(100):
             d[i%7][j%7] += 1

C:\TEMP>python -m timeit -s "from lf import test_lambda" "test_lambda()"
10 loops, best of 3: 80.3 msec per loop

C:\TEMP>python -m timeit -s "from lf import test_func" "test_func()"
10 loops, best of 3: 79.6 msec per loop

(the difference being less than the time variation I get, so it has no  
significance)

(b) I don't consider a one-line function so much ugly, but ugliness is  
subjective, so I won't comment further.

> Is there another way of achieving the same behaviour, that allow for
> pickling?

If you insist: Convert the defaultdict into a plain dict before pickling:  
plain_dict = dict(d)
and again into a defaultdict when unpickling: d=defaultdict(...);  
d.update(unpickled_dict)

-- 
Gabriel Genellina