[Numpy-discussion] possible bug: __array_wrap__ is not called during arithmetic operations in some cases

[Darren Dale]

On Sat, Mar 7, 2009 at 1:23 PM, Darren Dale <dsdale24 at gmail.com> wrote:

> On Sun, Feb 22, 2009 at 7:01 PM, Darren Dale <dsdale24 at gmail.com> wrote:
>
>> On Sun, Feb 22, 2009 at 6:35 PM, Darren Dale <dsdale24 at gmail.com> wrote:
>>
>>> On Sun, Feb 22, 2009 at 6:28 PM, Pierre GM <pgmdevlist at gmail.com> wrote:
>>>
>>>>
>>>> On Feb 22, 2009, at 6:21 PM, Eric Firing wrote:
>>>>
>>>> > Darren Dale wrote:
>>>> >> Does anyone know why __array_wrap__ is not called for subclasses
>>>> >> during
>>>> >> arithmetic operations where an iterable like a list or tuple
>>>> >> appears to
>>>> >> the right of the subclass? When I do "mine*[1,2,3]", array_wrap is
>>>> >> not
>>>> >> called and I get an ndarray instead of a MyArray. "[1,2,3]*mine" is
>>>> >> fine, as is "mine*array([1,2,3])". I see the same issue with
>>>> >> division,
>>>> >
>>>> > The masked array subclass does not show this behavior:
>>>>
>>>> Because MaskedArray.__mul__ and others are redefined.
>>>>
>>>> Darren, you can fix your problem by redefining MyArray.__mul__ as:
>>>>
>>>>     def __mul__(self, other):
>>>>         return np.ndarray.__mul__(self, np.asanyarray(other))
>>>>
>>>> forcing the second term to be a ndarray (or a subclass of). You can do
>>>> the same thing for the other functions (__add__, __radd__, ...)
>>>
>>>
>>> Thanks for the suggestion. I know this can be done, but ufuncs like
>>> np.multiply(mine,[1,2,3]) will still not work. Plus, if I reimplement these
>>> methods, I take some small performance hit. I've been putting a lot of work
>>> in lately to get quantities to work with numpy's stock ufuncs.
>>>
>>
>> I should point out:
>>
>> import numpy as np
>>
>> a=np.array([1,2,3,4])
>> b=np.ma.masked_where(a>2,a)
>> np.multiply([1,2,3,4],b) # yields a masked array
>> np.multiply(b,[1,2,3,4]) # yields an ndarray
>>
>>
> I'm not familiar with the numpy codebase, could anyone help me figure out
> where I should look to try to fix this bug? I've got a nice set of
> generators that work with nosetools to test all combinations of numerical
> dtypes, including combinations of scalars, arrays, and iterables of each
> type. In my quantities package, just testing multiplication yields 1031
> failures, all of which appear to be caused by this bug (#1026 on trak) or
> bug #826.



I finally managed to track done the source of this problem. _find_array_wrap
steps through the inputs, asking each of them for their __array_wrap__ and
binding it to wrap. If more than one input defines __array_wrap__, you enter
a block that selects one based on array priority, and binds it back to wrap.
The problem was when the first input defines array_wrap but the second one
does not. In that case, _find_array_wrap never bothered to rebind the
desired wraps[0] to wrap, so wrap remains Null or None, and wrap is what is
returned to the calling function.

I've tested numpy with this patch applied, and didn't see any regressions.
Would someone please consider committing it?

Thanks,
Darren

$ svn diff numpy/core/src/umath_ufunc_object.inc
Index: numpy/core/src/umath_ufunc_object.inc
===================================================================
--- numpy/core/src/umath_ufunc_object.inc       (revision 6569)
+++ numpy/core/src/umath_ufunc_object.inc       (working copy)
@@ -3173,8 +3173,10 @@
             PyErr_Clear();
         }
     }
+    if (np >= 1) {
+        wrap = wraps[0];
+    }
     if (np >= 2) {
-        wrap = wraps[0];
         maxpriority = PyArray_GetPriority(with_wrap[0],
                                         PyArray_SUBTYPE_PRIORITY);
         for (i = 1; i < np; ++i) {
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20090308/ae16e00e/attachment.html>