[Numpy-discussion] Preserving NumPy views when pickling

Feng Yu rainwoodman at gmail.com
Tue Oct 25 22:05:39 EDT 2016


Hi,

Just another perspective. base' and 'data' in PyArrayObject are two
separate variables.

base can point to any PyObject, but it is `data` that defines where
data is accessed in memory.

1. There is no clear way to pickle a pointer (`data`) in a meaningful
way. In order for `data` member to make sense we still need to
'readout' the values stored at `data` pointer in the pickle.

2. By definition base is not necessary a numpy array but it is just
some other object for managing the memory.

3. One can surely pickle the `base` object as a reference, but it is
useless if the data memory has been reconstructed independently during
unpickling.

4. Unless there is clear way to notify the referencing numpy array of
the new data pointer. There probably isn't.

BTW, is the stride information is lost during pickling, too? The
behavior shall probably be documented if not yet.

Yu

On Tue, Oct 25, 2016 at 5:29 PM, Robert Kern <robert.kern at gmail.com> wrote:
> On Tue, Oct 25, 2016 at 5:09 PM, Matthew Harrigan
> <harrigan.matthew at gmail.com> wrote:
>>
>> It seems pickle keeps track of references for basic python types.
>>
>> x = [1]
>> y = [x]
>> x,y = pickle.loads(pickle.dumps((x,y)))
>> x.append(2)
>> print(y)
>> >>> [[1,2]]
>>
>> Numpy arrays are different but references are forgotten after
>> pickle/unpickle.  Shared objects do not remain shared.  Based on the quote
>> below it could be considered bug with numpy/pickle.
>
> Not a bug, but an explicit design decision on numpy's part.
>
> --
> Robert Kern
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>



More information about the NumPy-Discussion mailing list