Multiprocessing, shared memory vs. pickled copies

Philip Semanchuk philip at
Tue Apr 5 19:43:08 CEST 2011

On Apr 5, 2011, at 12:58 PM, John Ladasky wrote:

> Hi Philip,
> Thanks for the reply.
> On Apr 4, 4:34 pm, Philip Semanchuk <phi... at> wrote:
>> So if you're going to use multiprocessing, you're going to use pickle, and you
>> need pickleable objects.
> OK, that's good to know.

But as Dan Stromberg pointed out, there are some pickle-free ways to communicate between processes using multiprocessing.

> This leads straight into my second question.  I THINK, without knowing
> for sure, that most user classes would pickle correctly by simply
> iterating through __dict__.  So, why isn't this the default behavior
> for Python?  Was the assumption that programmers would only want to
> pickle built-in classes?  

One can pickle user-defined classes:

>>> class Foo(object):
...     pass
>>> import pickle
>>> foo_instance = Foo()
>>> pickle.dumps(foo_instance)

And as Robert Kern pointed out, numpy arrays are also pickle-able.

>>> import numpy
>>> pickle.dumps(numpy.zeros(3))

As a side note, you should always use "new style" classes, particularly since you're exploring the details of Python class construction. "New" is a bit a of misnomer now, as "new" style classes were introduced in Python 2.2. They have been the status quo in Python 2.x for a while now and are the only choice in Python 3.x.

Subclassing object gives you a new style class:
   class Foo(object):

Not subclassing object (as you did in your example) gives you an old style class:
   class Foo:


More information about the Python-list mailing list