Multiprocessing, shared memory vs. pickled copies

John Ladasky ladasky at my-deja.com
Thu Apr 7 20:39:35 CEST 2011


On Apr 7, 10:44 am, Robert Kern <robert.k... at gmail.com> wrote:
> On 4/7/11 1:40 AM, John Ladasky wrote:
>
> > On Apr 5, 10:43 am, Philip Semanchuk<phi... at semanchuk.com>  wrote:
> >> And as Robert Kern pointed out, numpy arrays are also pickle-able.
>
> > OK, but SUBCLASSES of numpy.ndarray are not, in my hands, pickling as
> > I would expect.  I already have lots of code that is based on such
> > subclasses, and they do everything I want EXCEPT pickle correctly.  I
> > may have to direct this line of questions to the numpy discussion
> > group after all.
>
> Define the __reduce_ex__() method, not __getstate__(), __setstate__().
>
> http://docs.python.org/release/2.6.6/library/pickle.html#pickling-and...
>
> ndarrays are extension types, so they use that mechanism.

Thanks, Robert, as you can see, I got on that track shortly after I
posted my code example.  This is apparently NOT a numpy issue, it's an
issue for pickling all C extension types.

Is there a way to examine a Python object, and determine whether it's
a C extension type or not?  Or do you have to deduce that from the
documentation and/or the source code?

I started hunting through the numpy source code last night.  It's
complicated.  I haven't found the ndarray object definition yet.
Perhaps because I was looking through .py files, when I actually
should have been looking through .c files?



More information about the Python-list mailing list