[Pythonmac-SIG] Is this a reasonable way to do multiprocessing?
Lou Pecora
lou_boog2000 at yahoo.com
Tue Apr 19 19:48:21 CEST 2011
From: Nat Echols <nathaniel.echols at gmail.com>
To: Lou Pecora <lou_boog2000 at yahoo.com>
Cc: pythonmac-sig at python.org
Sent: Tue, April 19, 2011 1:39:27 PM
Subject: Re: [Pythonmac-SIG] Is this a reasonable way to do multiprocessing?
On Tue, Apr 19, 2011 at 10:11 AM, Lou Pecora <lou_boog2000 at yahoo.com> wrote:
But as many of you probably know that raises a pickle error. I did a little
>research and don't fully understand, but I thought of a way around this that is
>pedestrian, but it works on my toy example. I just make (deep) copies of my
>object and let a helper function call mainclass' aMethod. So I change the
code
(after my class definition) to,
>
This is overkill. I don't understand the precise technical explanation for the
error, but I think it just means that you can't pickle an object's method - only
the object itself (assuming it doesn't have features that break pickling). So
what I do is this:
import multiprocessing as MP
class mainclass (object) :
def __call__ (self, x) :
# do something
return result
-------------------------------------------------
I see, you use the __call__ function. Forgot completely about that. I think I
could change my class to handle that. I'll try it. Thanks very much. And
thanks for the info on cores and CPUs. I also have a 12 core Mac Pro so I can
get some speed up on that.
And my apologies to all for the bad formatting of the code in my message. I
should have replaced tabs with spaces.
-- Lou Pecora, my views are my own.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pythonmac-sig/attachments/20110419/7aa8b191/attachment-0001.html>
More information about the Pythonmac-SIG
mailing list