[Numpy-discussion] Extending ufunc signature syntax for matmul, frozen dimensions

[Matti Picus]

On 01/05/18 21:08, Marten van Kerkwijk wrote:
> Just for completeness: there are *four* gufuncs (matmat, matvec,
> vecmat, and vecvec).
>
> I remain torn about the best way forward. The main argument against
> using them inside matmul is that in order to decide which of the four
> to use, matmul has to have access to the `shape` of the arguments.
> This meants that means that `__array_ufunc__` cannot be used to
> override `matmul` (or `@`) for any object which does not have a shape.
>  From that perspective, multiple signatures is definitely a more
> elegant solution.
>
> An advantage of the separate solution is that they are useful
> independently of whether they are used internally in `matmul`; though,
> then again, with a multi-signature matmul, these would be trivially
> created as convenience functions.
>
> -- Marten
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
My goal is to solve issue #9028, "no way to override matmul/@ if 
__array_ufunc__ is set on other". Maybe I am too focused on that, it 
seems shape does not come into play here.

Given a call to matmul(self, other) it appears to me that the decision 
to commit to self.matmul or to call other.__array_ufunc__("__call__", 
self.matmul, ...) is independent of the shapes and needs only nin and nout.
In other words, the implementation of matmul becomes (simplified):

(matmul(self, other) called)->

     (use __array_ufunc__ and nin and nout to decide whether to defer to 
other's __array_ufunc__ via PyUFunc_CheckOverride which implements NEP13) ->

         (yes: call other.__array_ufunc__ as for any other ufunc),

         (no: call matmul like we currently do, no more __aray__ufunc__ 
testing needed)

So the two avenues I am trying are

1) make matmul a gufunc and then it will automatically use the 
__array_ufunc__ machinery without any added changes, but this requires 
expanding the meaning of a signature to allow dispatch

2) generalize the __array_ufunc__ machinery to handle some kind of 
wrapped function, the wrapper knows about nin and nout and calls 
PyUFunc_CheckOverride, which would allow matmul to work unchanged and 
might support other functions as well.

The issue of whether matmat, vecmat, matvec, and vecvec are functions, 
gufuncs accessible from userland, or not defined at all is secondary to 
the current issue of overriding matmul , we can decide that in the future.
If we do create ufuncs for these variants, calling a.vecmat(other) for 
instance will still resolve to other's __array_ufunc__ without needing 
to explore other's shape.

I probably misunderstood what you were driving at because I am so 
focused on this particular issue.

Matti