run a function in another processor in python

[geremy condra]

On Fri, Dec 10, 2010 at 4:42 AM, Astan Chee <astan.chee at gmail.com> wrote:
> On Fri, Dec 10, 2010 at 1:37 AM, Peter Otten <__peter__ at web.de> wrote:
>>
>> I can't replicate the crash. However, your problem looks like there is a
>> ready-to-use solution:
>>
>>
>> http://docs.python.org/library/multiprocessing.html#using-a-pool-of-workers
>>
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>
>
> Pool.map doesn't seem to be able to support multiple argument functions
> which is what I'm trying to do here. Any other suggestions?
> Thanks again

1. Post the real code you're using, and
2. Put the arguments you want in a tuple and pass that. As an example,
let's say I have the following function:

def my_func(x, y, z):
    return x + y * z

you could rewrite this as:

def my_func(*args):
    return args[0] + args[1] * args[2]


Here's a worked-out example:

#! /usr/bin/env python3

import multiprocessing

def my_func(x, y, z):
    return x + y * z

def my_func_wrapper(t):
    return my_func(*t)

# assume we can get an iterable over each argument
xs = [1, 2, 3, 4]
ys = [5, 6, 7, 8]
zs = [9, 1, 2, 3]

# set up the pool to match the number of CPUs
num_processes = multiprocessing.cpu_count()
pool = multiprocessing.Pool(num_processes)

#will call my_func(1, 5, 9), my_func(2, 6, 1), etc.
results = pool.map(my_func_wrapper, zip(xs, ys, zs))
print(results)


Interesting factoid: you can't seem to use lambda or a decorator to do
this, which would have been my first instinct. Pickle apparently
chokes, although marshall wouldn't.

Geremy Condra