[Numpy-discussion] replacing the mechanism for dispatching ufuncs

[Darren Dale]

What is the status of this proposal?

On Wed, Jun 22, 2011 at 6:56 PM, Mark Wiebe <mwwiebe at gmail.com> wrote:
> On Wed, Jun 22, 2011 at 4:57 PM, Darren Dale <dsdale24 at gmail.com> wrote:
>>
>> On Wed, Jun 22, 2011 at 1:31 PM, Mark Wiebe <mwwiebe at gmail.com> wrote:
>> > On Wed, Jun 22, 2011 at 7:34 AM, Lluís <xscript at gmx.net> wrote:
>> >>
>> >> Darren Dale writes:
>> >>
>> >> > On Tue, Jun 21, 2011 at 1:57 PM, Mark Wiebe <mwwiebe at gmail.com>
>> >> > wrote:
>> >> >> On Tue, Jun 21, 2011 at 12:36 PM, Charles R Harris
>> >> >> <charlesr.harris at gmail.com> wrote:
>> >> >>> How does the ufunc get called so it doesn't get caught in an
>> >> >>> endless
>> >> >>> loop?
>> >>
>> >> > [...]
>> >>
>> >> >> The function being called needs to ensure this, either by extracting
>> >> >> a
>> >> >> raw
>> >> >> ndarray from instances of its class, or adding a 'subok = False'
>> >> >> parameter
>> >> >> to the kwargs.
>> >>
>> >> > I didn't understand, could you please expand on that or show an
>> >> > example?
>> >>
>> >> As I understood the initial description and examples, the ufunc
>> >> overload
>> >> will keep being used as long as its arguments are of classes that
>> >> declare ufunc overrides (i.e., classes with the "_numpy_ufunc_"
>> >> attribute).
>> >>
>> >> Thus Mark's comment saying that you have to either transform the
>> >> arguments into raw ndarrays (either by creating new ones or passing a
>> >> view) or use the "subok = False" kwarg parameter to break a possible
>> >> overloading loop.
>> >
>> > The sequence of events is something like this:
>> > 1. You call np.sin(x)
>> > 2. The np.sin ufunc looks at x, sees the _numpy_ufunc_ attribute, and
>> > calls
>> > x._numpy_ufunc_(np.sin, x)
>> > 3. _numpy_ufunc_ uses np.sin.name (which is "sin") to get the correct
>> > my_sin
>> > function to call
>> > 4A. If my_sin called np.sin(x), we would go back to 1. and get an
>> > infinite
>> > loop
>> > 4B. If x is a subclass of ndarray, my_sin can call np.sin(x,
>> > subok=False),
>> > as this disables the subclass overloading mechanism.
>> > 4C. If x is not a subclass of ndarray, x needs to produce an ndarray,
>> > for
>> > instance it might have an x.arr property. Then it can call np.sin(x.arr)
>>
>> Ok, that seems straightforward and, for what its worth, it looks like
>> it would meet my needs. However, I wonder if the _numpy_func_
>> mechanism is the best approach. This is a bit sketchy, but why not do
>> something like:
>>
>> class Proxy:
>>
>>    def __init__(self, ufunc, *args):
>>        self._ufunc = ufunc
>>        self._args = args
>>
>>    def __call__(self, func):
>>        self._ufunc._registry[tuple(type(arg) for arg in self._args)] =
>> func
>>        return func
>>
>>
>> class UfuncObject:
>>
>>    ...
>>
>>    def __call__(self, *args, **kwargs):
>>        func = self._registry.get(tuple(type(arg) for arg in args), None)
>>        if func is None:
>>            raise TypeError
>>        return func(*args, **kwargs)
>>
>>    def register(self, *args):
>>        return Proxy(self, *args)
>>
>>
>> @np.sin.register(Quantity)
>> def sin(pq):
>>    if pq.units != degrees:
>>        pq = pq.rescale(degrees)
>>    temp = np.sin(pq.view(np.ndarray))
>>    return Quantity(temp, copy=False)
>>
>> This way, classes don't have to implement special methods to support
>> ufunc registration, special attributes to claim primacy in ufunc
>> registration lookup, special versions of the functions for each numpy
>> ufunc, *and* the logic to determine whether the combination of
>> arguments is supported. By that I mean, if I call np.sum with a
>> quantity and a masked array, and Quantity wins the __array_priority__
>> competition, then I also need to check that my special sum function(s)
>> know how to operate on that combination of inputs. With the decorator
>> approach, I just need to implement the special versions of the ufuncs,
>> and the decorators handle the logic of knowing what combinations of
>> arguments are supported.
>>
>> It might be worth considering using ABCs for registration and have
>> UfuncObject use isinstance to determine the appropriate special
>> function to call.
>
> The thing I'm not sure about with this idea is how to do something general
> to modify all ufuncs for a particular class in one swoop. Having to
> individually override everything would be a hassle, I think. I also don't
> think this approach saves very much effort compared to the _numpy_ufunc_
> call approach, checking the types and raising NotImplemented if they aren't
> what's wanted is pretty much the only difference, and that check could still
> be handled by a decorator like you're showing.
> -Mark
>
>>
>> Darren
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>