[Numpy-discussion] immutable numpy arrays

[Geoffrey Irving]

On Wed, Dec 17, 2008 at 4:28 PM, Robert Kern <robert.kern at gmail.com> wrote:
> On Wed, Dec 17, 2008 at 17:45, Geoffrey Irving <irving at naml.us> wrote:
>> On Wed, Dec 17, 2008 at 3:34 PM, Robert Kern <robert.kern at gmail.com> wrote:
>>> On Wed, Dec 17, 2008 at 16:51, Geoffrey Irving <irving at naml.us> wrote:
>>>> On Wed, Dec 17, 2008 at 2:24 PM, Robert Kern <robert.kern at gmail.com> wrote:
>>>>> On Wed, Dec 17, 2008 at 15:52, Geoffrey Irving <irving at naml.us> wrote:
>>>>>> Currently numpy arrays are either writable or unwritable, but
>>>>>> unwritable arrays can still be changed through other copies.  This
>>>>>> means that when a numpy array is passed into an interface that
>>>>>> requires immutability for safety reasons, a copy always has to be
>>>>>> made.
>>>>>>
>>>>>> One way around this would be to add a NPY_IMMUTABLE flag signifying
>>>>>> that the contents of the array will never change through other copies.
>>>>>
>>>>> This is not possible to guarantee. With the __array_interface__, I can
>>>>> make a numpy array point at any addressable memory without its
>>>>> knowledge. We can even mutate "immutable" str objects, too.
>>>>
>>>> In python __array_interface__ just returns a big integer representing
>>>> a pointer which can't be used for anything.
>>>
>>> I can (and do) *make* an array from Python given an
>>> __array_interface__ with that pointer. See numpy/lib/stride_trick.py
>>> in numpy 1.2 for an example.
>>
>> Ah.  Yes, that certainly precludes complete safety.
>>
>> I don't think it precludes the usefulness of an immutable flag though,
>> just like it doesn't preclude the usefulness of the writeable flag.
>> The stride_tricks.py code is already well-behaved: it doesn't turn
>> unwriteable arrays into writeable arrays.  It certainly could, but
>> this is analogous to ctypes or untrusted C code.
>
> It just seems to me to be another complication that does not provide
> any guarantees. You say "Currently numpy arrays are either writable or
> unwritable, but unwritable arrays can still be changed through other
> copies." Adding an immutable flag would just change that to "Currently
> numpy arrays are either mutable or immutable, but immutable arrays can
> still be changed through other copies." Basically, the writable flag
> is intended to indicate your use case. It can be circumvented, but the
> same methods of circumvention can be applied to any set of flags.

The point of an immutable array would be that _can't_ be changed
through other copies except through broken C code (or the ctypes /
__array_interface__ equivalents), so it's not correct to say that it's
the same as unwriteable.  It's the same distinction as C++ const vs.
Java final.  Immutability is already a common notion in python, e.g.,
list vs. tuple and set vs. frozenset, and it's unfortunate that numpy
doesn't have an equivalent.

However, if you agree that even _with_ the guarantee it's not a useful
concept, I'm happy to drop it.

As far as the lack of guarantee, here's some python code that modifies
1.  Just because I can write it doesn't mean that we should tell
people not to trust the values of small integers. :)

    import sys
    from numpy import *

    class Evil:
        size = int(log(float(sys.maxint+1))/log(2)+1)/8
        __array_interface__ = {
            'shape' : (1,),
            'typestr' : '<i%d'%size,
            'data' : (id(1)+2*size, 0) }

    asarray(Evil())[0] = 2
    assert 1 == 2

Geoffrey