Re: [Numpy-discussion] numarray unfriendly to user defined types

On Thu, 2003-09-18 at 19:18, Tim Hochberg wrote:

Sorry for the delay; I was pretty much shut down by hurricane Isabelle from the end of last week.

I hit and bypassed this issue trying to port MA.

I've run into another issue in my attempt to transition over to numarray: it's less friendly to user defined types than Numeric. I think this is mainly accidental friednliness on Numeric's part, but it's handy nonetheless. The attached file illustrates the issue. Given some object, in this case *zero*, that a numarray array does not now how to handle, it ends up raising an exception instead of giving the object a chance to try. Thus ``zero + numarray.arange(5)`` works while ``numarray.arange(5)

• zero`` fails, since in the first case Zero.__add__ is called first,

while in the second NumArray.__add__ is called first.

This should probably be fixed, but it's not clear what the best way is. My first thought was to always give the other object a chance to use __rop__ first. This is simple and easy to explain, but fails miserably in the common case of multiplying a list and an array. Just to be concrete, __mul__ would be replaced by::

def __mul__(self, operand):
    try:
        return operand.__rmul__(self)
    except:
        return ufunc.multiply(self, operand)

Next thought is to catch exceptions when they occur in numarray and then give the other operand a chance::

def __mul__(self, operand):
    try:
       return ufunc.multiply(self, operand)
    except:
        return operand.__rmul__(self)

This appears like it would fix my particular problem, but still is not ideal. Since numarray is the base of libraries that it will know nothing about, it should defer to classes it doesn't know about whenever possible.

That does sound like the right heuristic and is echoed (somewhat) in the core language.

Otherewise it's not possible (or maybe just hard) to create new classes that try to be clever, but still interact with numarray in a reasonable way.

Looking in Python's Objects/abstract.c, the function binop1 shows one way to squirm around this issue: subclass from NumArray. It would be interesting to know how Numeric does it.

In my case I'm thinking of proxy objects that don't do some computations till they are actually required. So my current thinking is that __mul__ and friends would be best implemented as::

def __mul__(self, operand):
    if not isinstance(operand, knownTypes):

If the "not" belongs there, I'm lost.

        try:
              return ufunc.multiply(self, operand)
        except:
            pass
    return operand.__rmul__(self)

Where knownTypes is something like (int, long, float, complex, tuple, list).

Anyway, that's my excessively long two cents on this.

I agree with your heuristic, but I am leery of putting an unqualified try/except in front of the numarray ufunc code. On the other hand, qualifying it seems too complicated.

What about adding a tuple of types to be deferred to?

def __mul__(self, operand): if isinstance(operand, _numarray_deferred_types): operand.__rmul__(self) else: self.__mul__(operand)

Then other libraries could register classes with something like:

numarray.defer_to(my_class)

which is slightly painful, but avoids masking the ufunc exceptions and only needs to be done once for each library base class. How does that sound?

Todd

-tim

---

import numarray as na import Numeric as np

class Zero: def __add__(self, other): return other __radd__ = __add__ zero = Zero()

#~ print zero + np.arange(5) #~ print np.arange(5) + zero #~ print zero + na.arange(5) #~ print na.arange(5) + zero

a = na.arange(5)

import copy copy.deepcopy(a)