[Pythonmac-SIG] Is this a reasonable way to do multiprocessing?

Lou Pecora lou_boog2000 at yahoo.com
Tue Apr 19 19:48:21 CEST 2011


From: Nat Echols <nathaniel.echols at gmail.com>
To: Lou Pecora <lou_boog2000 at yahoo.com>
Cc: pythonmac-sig at python.org
Sent: Tue, April 19, 2011 1:39:27 PM
Subject: Re: [Pythonmac-SIG] Is this a reasonable way to do multiprocessing?

On Tue, Apr 19, 2011 at 10:11 AM, Lou Pecora <lou_boog2000 at yahoo.com> wrote:

But as many of you probably know that raises a pickle error.  I did a little
>research and don't fully understand, but I thought of a way around this that is
>pedestrian, but it works on my toy example. I just make (deep) copies of my
>object and let a helper function call mainclass'  aMethod. So I change the 
code 
(after my class definition) to,
>

This is overkill.  I don't understand the precise technical explanation for the 
error, but I think it just means that you can't pickle an object's method - only 
the object itself (assuming it doesn't have features that break pickling).  So 
what I do is this:

import multiprocessing as MP

class mainclass (object) :
  def __call__ (self, x) :
     # do something
     return result

-------------------------------------------------

I see, you use the __call__ function.  Forgot completely about that.  I think I 
could change my class to handle that.  I'll try it.  Thanks very much.  And 
thanks for the info on cores and CPUs.  I also have a 12 core Mac Pro so I can 
get some speed up on that.

And my apologies to all for the bad formatting of the code in my message.  I 
should have replaced tabs with spaces.

-- Lou Pecora, my views are my own.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pythonmac-sig/attachments/20110419/7aa8b191/attachment-0001.html>


More information about the Pythonmac-SIG mailing list