[Numpy-discussion] replacing the mechanism for dispatching ufuncs

[Darren Dale]

On Wed, Jun 22, 2011 at 1:31 PM, Mark Wiebe <mwwiebe at gmail.com> wrote:
> On Wed, Jun 22, 2011 at 7:34 AM, Lluís <xscript at gmx.net> wrote:
>>
>> Darren Dale writes:
>>
>> > On Tue, Jun 21, 2011 at 1:57 PM, Mark Wiebe <mwwiebe at gmail.com> wrote:
>> >> On Tue, Jun 21, 2011 at 12:36 PM, Charles R Harris
>> >> <charlesr.harris at gmail.com> wrote:
>> >>> How does the ufunc get called so it doesn't get caught in an endless
>> >>> loop?
>>
>> > [...]
>>
>> >> The function being called needs to ensure this, either by extracting a
>> >> raw
>> >> ndarray from instances of its class, or adding a 'subok = False'
>> >> parameter
>> >> to the kwargs.
>>
>> > I didn't understand, could you please expand on that or show an example?
>>
>> As I understood the initial description and examples, the ufunc overload
>> will keep being used as long as its arguments are of classes that
>> declare ufunc overrides (i.e., classes with the "_numpy_ufunc_"
>> attribute).
>>
>> Thus Mark's comment saying that you have to either transform the
>> arguments into raw ndarrays (either by creating new ones or passing a
>> view) or use the "subok = False" kwarg parameter to break a possible
>> overloading loop.
>
> The sequence of events is something like this:
> 1. You call np.sin(x)
> 2. The np.sin ufunc looks at x, sees the _numpy_ufunc_ attribute, and calls
> x._numpy_ufunc_(np.sin, x)
> 3. _numpy_ufunc_ uses np.sin.name (which is "sin") to get the correct my_sin
> function to call
> 4A. If my_sin called np.sin(x), we would go back to 1. and get an infinite
> loop
> 4B. If x is a subclass of ndarray, my_sin can call np.sin(x, subok=False),
> as this disables the subclass overloading mechanism.
> 4C. If x is not a subclass of ndarray, x needs to produce an ndarray, for
> instance it might have an x.arr property. Then it can call np.sin(x.arr)

Ok, that seems straightforward and, for what its worth, it looks like
it would meet my needs. However, I wonder if the _numpy_func_
mechanism is the best approach. This is a bit sketchy, but why not do
something like:

class Proxy:

    def __init__(self, ufunc, *args):
        self._ufunc = ufunc
        self._args = args

    def __call__(self, func):
        self._ufunc._registry[tuple(type(arg) for arg in self._args)] = func
        return func


class UfuncObject:

    ...

    def __call__(self, *args, **kwargs):
        func = self._registry.get(tuple(type(arg) for arg in args), None)
        if func is None:
            raise TypeError
        return func(*args, **kwargs)

    def register(self, *args):
        return Proxy(self, *args)


@np.sin.register(Quantity)
def sin(pq):
    if pq.units != degrees:
        pq = pq.rescale(degrees)
    temp = np.sin(pq.view(np.ndarray))
    return Quantity(temp, copy=False)

This way, classes don't have to implement special methods to support
ufunc registration, special attributes to claim primacy in ufunc
registration lookup, special versions of the functions for each numpy
ufunc, *and* the logic to determine whether the combination of
arguments is supported. By that I mean, if I call np.sum with a
quantity and a masked array, and Quantity wins the __array_priority__
competition, then I also need to check that my special sum function(s)
know how to operate on that combination of inputs. With the decorator
approach, I just need to implement the special versions of the ufuncs,
and the decorators handle the logic of knowing what combinations of
arguments are supported.

It might be worth considering using ABCs for registration and have
UfuncObject use isinstance to determine the appropriate special
function to call.

Darren