Hrvoje Niksic wrote:

Calvin Spealman wrote:

...

I don't think it would be unreasonable to consider either 1) making functools.partial picklable (I don't know how feasible this is)

It's not only feasible, but quite easy and, I think, useful. A "partial" instance is a simple triplet of (function, args, kwds), and it can be pickled as such. For example:

...

...

...

import copy_reg, functools def _reconstruct_partial(f, args, kwds): ... return functools.partial(f, *args, **(kwds or {})) ... def _reduce_partial(p): ... return _reconstruct_partial, (p.func, p.args, p.keywords) ... copy_reg.pickle(functools.partial, _reduce_partial)

Test:

...

...

...

import operator, cPickle as cp p = functools.partial(operator.add, 3) p(10) 13 cp.dumps(p)

'c__main__\n_reconstruct_partial\np1\n(coperator\nadd\np2\n(I3\ntp3\nNtRp4\n.'

...

...

...

p2 = cp.loads(_) p2(10) 13

Iedally this should be implemented in the functools.partial object itself. Confirmed:

from multiprocessing import Pool def power (x, pwr=2): return x**pwr import functools run_test = functools.partial (power, pwr=3) import copy_reg, functools def _reconstruct_partial(f, args, kwds): return functools.partial(f, *args, **(kwds or {})) def _reduce_partial(p): return _reconstruct_partial, (p.func, p.args, p.keywords) copy_reg.pickle(functools.partial, _reduce_partial) if __name__ == "__main__": pool = Pool() cases = [3,4,5] results = pool.map (run_test, cases) print results $python test_multi.py [27, 64, 125]