Multiprocessing, shared memory vs. pickled copies
philip at semanchuk.com
Tue Apr 5 19:43:08 CEST 2011
On Apr 5, 2011, at 12:58 PM, John Ladasky wrote:
> Hi Philip,
> Thanks for the reply.
> On Apr 4, 4:34 pm, Philip Semanchuk <phi... at semanchuk.com> wrote:
>> So if you're going to use multiprocessing, you're going to use pickle, and you
>> need pickleable objects.
> OK, that's good to know.
But as Dan Stromberg pointed out, there are some pickle-free ways to communicate between processes using multiprocessing.
> This leads straight into my second question. I THINK, without knowing
> for sure, that most user classes would pickle correctly by simply
> iterating through __dict__. So, why isn't this the default behavior
> for Python? Was the assumption that programmers would only want to
> pickle built-in classes?
One can pickle user-defined classes:
>>> class Foo(object):
>>> import pickle
>>> foo_instance = Foo()
And as Robert Kern pointed out, numpy arrays are also pickle-able.
>>> import numpy
As a side note, you should always use "new style" classes, particularly since you're exploring the details of Python class construction. "New" is a bit a of misnomer now, as "new" style classes were introduced in Python 2.2. They have been the status quo in Python 2.x for a while now and are the only choice in Python 3.x.
Subclassing object gives you a new style class:
Not subclassing object (as you did in your example) gives you an old style class:
More information about the Python-list