[Tutor] differences between map and partial?

[Peter Otten]

source liu wrote:

> Hi, Danny,
> 
> 
> Thank you for your reply
> 
> I checked the mail, the attachment is attached, i don't know why you
> can't see it ( I change the extension of .tar.gz to .tar.gz_src, as
> gmail can't pass the security check of the .tar.gz)

They are stripped off by the mailing list software, so don't bother trying.

> from multiprocessing import Pool

>     p=Pool(processes=4);

>     print p.map(partial(file_op,lineop=unity),input)
>     #print p.map(lambda x:file_op(x,unity), input)

The missing part of information was that you are using multiprocessing (The 
traceback would also have shown the problem).
Multiprocessing uses pickle to pass data around between processes, and 
anonymous functions (aka lambdas) cannot be pickled. Instead you have to use 
an ordinary function defined on the module level

def whatever(x):
    return file_op(x, unity)

p.map(whatever, input)

For these pickle need not pass the code, it just remembers the module and 
function name which are then used to look up the function during unpickling.

As you found out functools.partial() can be pickled, too, and thus works 
when all of its arguments can be pickled (in particular its first argument 
has to be a global function rather than a local one or a lambda).