<div dir="ltr">An issue was raised yesterday in github, regarding <font face="monospace, monospace">np.may_share_memory</font> when run on a class exposing an array using the <font face="monospace, monospace">__array__</font> method. You can check the details here:<div><br></div><div><a href="https://github.com/numpy/numpy/issues/5604" target="_blank">https://github.com/numpy/numpy/issues/5604</a></div><div><br></div><div>Looking into it, I found out that NumPy doesn't really treat objects exposing <font face="monospace, monospace">__array__</font>,, <font face="monospace, monospace">__array_interface__</font>, or <font face="monospace, monospace">__array_struct__</font> as if they were proper arrays:</div><div><ol><li>When converting these objects to arrays using <font face="monospace, monospace">PyArray_Converter,</font> if the arrays returned by any of the array interfaces is not C contiguous, aligned, and writeable, a copy that is will be made. Proper arrays and subclasses are passed unchanged. This is the source of the error reported above.</li><li>When converting these objects using PyArray_OutputConverter, as well as in similar code in the ufucn machinery, anything other than a proper array or subclass raises an error. This means that, contrary to what the docs on subclassing say, see below, you cannot use an object exposing the array interface as an output parameter to a ufunc</li></ol><div>The following classes can be used to test this behavior:</div></div><div><br></div><div><div><font face="monospace, monospace">class Foo:</font></div><div><font face="monospace, monospace"> def __init__(self, arr):</font></div><div><font face="monospace, monospace"> self.arr = arr</font></div><div><font face="monospace, monospace"> def __array__(self):</font></div><div><font face="monospace, monospace"> return self.arr</font></div><div><font face="monospace, monospace"> </font></div><div><font face="monospace, monospace">class Bar:</font></div><div><font face="monospace, monospace"> def __init__(self, arr):</font></div><div><font face="monospace, monospace"> self.arr = arr</font></div><div><font face="monospace, monospace"> self.__array_interface__ = arr.__array_interface__</font></div><div><font face="monospace, monospace"> </font></div><div><font face="monospace, monospace">class Baz:</font></div><div><font face="monospace, monospace"> def __init__(self, arr):</font></div><div><font face="monospace, monospace"> self.arr = arr</font></div><div><font face="monospace, monospace"> self.__array_struct__ = arr.__array_struct__</font></div></div><div><br></div><div>They all behave the same with these examples:</div><div><br></div><div><div><font face="monospace, monospace">>>> a = Foo(np.ones(5))</font></div><div><font face="monospace, monospace">>>> np.add(a, a)</font></div><div><font face="monospace, monospace">array([ 2., 2., 2., 2., 2.])</font></div><div><font face="monospace, monospace">>>> np.add.accumulate(a)</font></div><div><font face="monospace, monospace">array([ 1., 2., 3., 4., 5.])</font></div><div><font face="monospace, monospace">>>> np.add(a, a, out=a)</font></div><div><font face="monospace, monospace">Traceback (most recent call last):</font></div><div><font face="monospace, monospace"> File "<stdin>", line 1, in <module></font></div><div><font face="monospace, monospace">TypeError: return arrays must be of ArrayType</font></div><div><font face="monospace, monospace">>>> np.add.accumulate(a, out=a)</font></div><div><font face="monospace, monospace">Traceback (most recent call last):</font></div><div><font face="monospace, monospace"> File "<stdin>", line 1, in <module></font></div><div><font face="monospace, monospace">TypeError: output must be an array</font></div></div><div><br></div><div>I think this should be changed, and whatever gets handed by this methods/interfaces be treated as if it were an array or subclass of it. This is actually what the docs on subclassing say about <font face="monospace, monospace">__array__</font> here:</div><div><br></div><div><a href="http://docs.scipy.org/doc/numpy/reference/arrays.classes.html#numpy.class.__array__">http://docs.scipy.org/doc/numpy/reference/arrays.classes.html#numpy.class.__array__</a><br></div><div><br></div><div>This also seems to contradict a rather cryptic comment in the code of <font face="monospace, monospace">PyArray_GetArrayParamsFromObject</font>, which is part of the call sequence of this whole mess, see here:</div><div><br></div><div><a href="https://github.com/numpy/numpy/blob/maintenance/1.9.x/numpy/core/src/multiarray/ctors.c#L1495">https://github.com/numpy/numpy/blob/maintenance/1.9.x/numpy/core/src/multiarray/ctors.c#L1495</a><br></div><div><br></div><div><div><font face="monospace, monospace">/*</font></div><div><font face="monospace, monospace"> * If op supplies the __array__ function.</font></div><div><font face="monospace, monospace"> * The documentation says this should produce a copy, so</font></div><div><font face="monospace, monospace"> * we skip this method if writeable is true, because the intent</font></div><div><font face="monospace, monospace"> * of writeable is to modify the operand.</font></div><div><font face="monospace, monospace"> * XXX: If the implementation is wrong, and/or if actual</font></div><div><font face="monospace, monospace"> * usage requires this behave differently,</font></div><div><font face="monospace, monospace"> * this should be changed!</font></div><div><font face="monospace, monospace"> */</font></div></div><div><br></div><div>There has already been some discussion in the issue linked above, but I would appreciate any other thoughts on the idea of treating objects with some form of array interface as if they were arrays. Does it need a deprecation cycle? Is there some case I am not considering where this could go horribly wrong?</div><div><br></div><div>Jaime</div><div><br></div><div>-- <br><div>(\__/)<br>( O.o)<br>( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.</div>
</div></div>