[SciPy-Dev] Enabling use of scipy.special from Numba

[Joshua Wilson]

Ok, returning to the special functions in Numba topic: here's a small
repo that generates overloads for a few special functions:

https://github.com/person142/numba_special/tree/master/numba_special

Taking Nathaniel's point that coming up with an ad-hoc metadata format
is probably not a good idea, it instead uses a little bit of Cython to
pull out the function pointers:

https://github.com/person142/numba_special/blob/master/numba_special/function_pointers.pyx

and then you grab those on the Numba side to create the overloads:

https://github.com/person142/numba_special/blob/master/numba_special/numba_overloads.py

You can see that the overloads are working as expected in the tests:

https://github.com/person142/numba_special/blob/master/numba_special/tests/test_functions.py

(Note that the development version of Numba is required because of
this issue: https://github.com/numba/numba/issues/4133)

Of course the pointers/overloads that you're creating are highly
structured, so you can just generate the necessary glue code:

https://github.com/person142/numba_special/blob/master/generate_overloads.py

though it only handles simple function signatures right now. All in
all I think it works pretty well-you only use 100% officially
supported pieces of all the libraries involved. I'll probably work on
extending it more in the next couple of days.

On Thu, Jun 20, 2019 at 5:31 PM Nathaniel Smith <njs at pobox.com> wrote:
>
> On Thu, Jun 20, 2019 at 1:41 PM Stanley Seibert <sseibert at anaconda.com> wrote:
> >
> > Well, we'd prefer not to have an unnecessary indirection (and the overhead of making a nested call to the scalar inside the length 1 loop) since the user is doing their own iteration in Numba, but if that's the only option, we can certainly work around it.
>
> Well, it's computers, so there's never just one option :-). But if you
> can consume existing ufuncs and get 95% of the performance, then it
> might not be worth building custom infrastructure to expose
> scipy.special's raw scalar functions + an ad hoc metadata format + ad
> hoc tools to consume the metadata. Intuitively it seems like a few
> extra integer operations at entry and exit might not be noticeable at
> all compared to the cost of computing a bessel function or whatever,
> but I haven't checked.
>
> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev