[Numpy-discussion] numarray unfriendly to user defined types

[Todd Miller]

On Thu, 2003-09-18 at 19:18, Tim Hochberg wrote:
> 
Sorry for the delay;  I was pretty much shut down by hurricane Isabelle
from the end of last week.

I hit and bypassed this issue trying to port MA.

> I've run into another issue in my attempt to transition over to 
> numarray: it's less friendly to user defined types than Numeric. I think 
> this is mainly accidental friednliness on Numeric's part, but it's handy 
> nonetheless. The attached file illustrates the issue. Given some object, 
> in this case *zero*, that a numarray array does not now how to handle, 
> it ends up raising an exception instead of giving the object a chance to 
> try. Thus ``zero + numarray.arange(5)`` works while ``numarray.arange(5) 
> + zero`` fails, since in the first case Zero.__add__ is called first, 
> while in the second NumArray.__add__ is called first.
> 
> This should probably be fixed, but it's not clear what the best way is. 
> My first thought was to always give the other object a chance to use 
> __rop__ first. This is simple and easy to explain, but fails miserably 
> in the common case of multiplying a list and an array. Just to be 
> concrete, __mul__ would be replaced by::
> 
>     def __mul__(self, operand):
>         try:
>             return operand.__rmul__(self)
>         except:
>             return ufunc.multiply(self, operand)
> 
> Next thought is to catch exceptions when they occur in numarray and then 
> give the other operand a chance::
> 
>     def __mul__(self, operand):
>         try:
>            return ufunc.multiply(self, operand)
>         except:
>             return operand.__rmul__(self)
> 
> This appears like it would fix my particular problem, but still is not 
> ideal. Since numarray is the base of libraries that it will know nothing 
> about, it should defer to classes it doesn't know about whenever 
> possible. 

That does sound like the right heuristic and is echoed (somewhat) in the
core language. 

> Otherewise it's not possible (or maybe just hard)  to create  
> new classes that try to be clever, but still interact with numarray in a 
> reasonable way. 

Looking in Python's Objects/abstract.c, the function binop1 shows one
way to squirm around this issue:  subclass from NumArray.   It would be
interesting to know how Numeric does it.  

> In my case I'm thinking of proxy objects that don't do 
> some computations till they are actually required. So my current 
> thinking is that __mul__ and friends would be best implemented as::
> 
>     def __mul__(self, operand):
>         if not isinstance(operand, knownTypes):

If the "not" belongs there, I'm lost.

>             try:
>                   return ufunc.multiply(self, operand)
>             except:
>                 pass
>         return operand.__rmul__(self)
> 
> Where knownTypes is something like (int, long, float, complex, tuple, list).
> 
> Anyway, that's my excessively long two cents on this.
> 
I agree with your heuristic,  but I am leery of putting an unqualified
try/except in front of the numarray ufunc code.  On the other hand,
qualifying it seems too complicated.  

What about adding a tuple of types to be deferred to?

	def __mul__(self, operand):
		if isinstance(operand, _numarray_deferred_types):
			operand.__rmul__(self)
		else:
			self.__mul__(operand)

Then other libraries could register classes with something like:

numarray.defer_to(my_class)

which is slightly painful, but avoids masking the ufunc exceptions and
only needs to be done once for each library base class.  How does that
sound?

Todd

> -tim
> 
> 
> 
> 
> ----
> 

> import numarray as na
> import Numeric as np
> 
> class Zero:
>     def __add__(self, other):
>         return other
>     __radd__ = __add__
> zero = Zero()
>     
> #~ print zero + np.arange(5)
> #~ print np.arange(5) + zero
> #~ print zero + na.arange(5) 
> #~ print na.arange(5) + zero
> 
> a = na.arange(5)
> 
> import copy
> copy.deepcopy(a)
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB