[Numpy-discussion] replacing the mechanism for dispatching ufuncs

[Mark Wiebe]

On Tue, Jun 21, 2011 at 12:36 PM, Charles R Harris <
charlesr.harris at gmail.com> wrote:

>
>
> On Mon, Jun 20, 2011 at 12:32 PM, Mark Wiebe <mwwiebe at gmail.com> wrote:
>
>> NumPy has a mechanism built in to allow subclasses to adjust or override
>> aspects of the ufunc behavior. While this goal is important, this mechanism
>> only allows for very limited customization, making for instance the masked
>> arrays unable to work with the native ufuncs in a full and proper way. I
>> would like to deprecate the current mechanism, in particular
>> __array_prepare__ and __array_wrap__, and introduce a new method I will
>> describe below. If you've ever used these mechanisms, please review this
>> design to see if it meets your needs.
>>
>>
>>
> The current approach is at a dead end, so something better needs to be
> done.
>
>
>> Any class type which would like to override its behavior in ufuncs would
>> define a method called _numpy_ufunc_, and optionally an attribute
>> __array_priority__ as can already be done. The class which wins the priority
>> battle gets its _numpy_ufunc_ function called as follows:
>>
>> return arr._numpy_ufunc_(current_ufunc, *args, **kwargs)
>>
>>
>> To support this overloading, the ufunc would get a new support method,
>> result_type, and there would be a new global function, broadcast_empty_like.
>>
>> The function ufunc.empty_like behaves like the global np.result_type, but
>> produces the output type or a tuple of output types specific to the ufunc,
>> which may follow a different convention than regular arithmetic type
>> promotion. This allows for a class to create an output array of the correct
>> type to pass to the ufunc if it needs to be different than the default.
>>
>> The function broadcast_empty_like is just like empty_like, but takes a
>> list or tuple of arrays which are to be broadcast together for producing the
>> output, instead of just one.
>>
>>
> How does the ufunc get called so it doesn't get caught in an endless loop?
> I like the proposed method if it can also be used for classes that don't
> subclass ndarray. Masked array, for instance, should probably not subclass
> ndarray.
>

The function being called needs to ensure this, either by extracting a raw
ndarray from instances of its class, or adding a 'subok = False' parameter
to the kwargs. Supporting objects that aren't ndarray subclasses is one of
the purposes for this approach, and neither of my two example cases
subclassed ndarray.

-Mark


>
>
>> Thanks,
>> Mark
>>
>>
>> A simple class which overrides the ufuncs might look as follows:
>>
>> def sin(ufunc, *args, **kwargs):
>>     # Convert degrees to radians
>>     args[0] = np.deg2rad(args[0])
>>     # Return a regular array, since the result is not in degrees
>>     return ufunc(*args, **kwargs)
>>
>> class MyDegreesClass:
>>     """Array-like object with a degrees unit"""
>>
>>     def __init__(arr):
>>         self.arr = arr
>>
>>     def _numpy_ufunc_(ufunc, *args, **kwargs):
>>         override = globals().get(ufunc.name)
>>         if override:
>>             return override(ufunc, *args, **kwargs)
>>         else:
>>             raise TypeError, 'ufunc %s incompatible with MyDegreesClass' %
>> ufunc.name
>>
>>
>>
>> A more complex example will be something like this:
>>
>> def my_general_ufunc(ufunc, *args, **kwargs):
>>     # Extract the 'out' argument. This only supports ufuncs with
>>     # one output, currently.
>>     out = kwargs.get('out')
>>     if len(args) > ufunc.nin:
>>         if out is None:
>>             out = args[ufunc.nin]
>>         else:
>>             raise ValueError, "'out' given as both a position and keyword
>> argument"
>>
>>     # Just want the inputs from here on
>>     args = args[:ufunc.nin]
>>
>>     # Strip out MyArrayClass, but allow operations with regular ndarrays
>>     raw_in = []
>>     for a in args:
>>         if isinstance(a, MyArrayClass):
>>             raw_in.append(a.arr)
>>         else:
>>             raw_in.append(a)
>>
>>     # Allocate the output array
>>     if not out is None:
>>         if isinstance(out, MyArrayClass):
>>             raise TypeError, "'out' must have type MyArrayClass"
>>     else:
>>         # Create the output array, obeying the 'order' parameter,
>>         # but disallowing subclasses
>>         out = np.broadcast_empty_like([args,
>>                                 order=kwargs.get('order'),
>>                                 dtype=ufunc.result_type(args),
>>                                 subok=False)
>>
>>     # Override the output argument
>>     kwargs['out'] = out.arr
>>
>>     # Call the ufunc
>>     ufunc(*args, **kwargs)
>>
>>     # Return the output
>>     return out
>>
>> class MyArrayClass:
>>     def __init__(arr):
>>         self.arr = arr
>>
>>     def _numpy_ufunc_(ufunc, *args, **kwargs):
>>         override = globals().get(ufunc.name)
>>         if override:
>>             return override(ufunc, *args, **kwargs)
>>         else:
>>             return my_general_ufunc(ufunc, *args, **kwargs)
>>
>>
> Chuck
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20110621/a875eef0/attachment.html>