[SciPy-Dev] Enabling use of scipy.special from Numba

[Joshua Wilson]

Final update-an initial release of the package is up on PyPI:

https://pypi.org/project/numba-special/

There's more work to be done, but you can see a list what can
currently be used from Numba here:

https://numba-special.readthedocs.io/en/latest/numba_special.html

On Thu, Jul 4, 2019 at 4:00 PM Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
>
>
> On Thu, Jul 4, 2019 at 12:23 AM Joshua Wilson <josh.craig.wilson at gmail.com> wrote:
>>
>> Ok, returning to the special functions in Numba topic: here's a small
>> repo that generates overloads for a few special functions:
>>
>> https://github.com/person142/numba_special/tree/master/numba_special
>>
>> Taking Nathaniel's point that coming up with an ad-hoc metadata format
>> is probably not a good idea, it instead uses a little bit of Cython to
>> pull out the function pointers:
>>
>> https://github.com/person142/numba_special/blob/master/numba_special/function_pointers.pyx
>>
>> and then you grab those on the Numba side to create the overloads:
>>
>> https://github.com/person142/numba_special/blob/master/numba_special/numba_overloads.py
>>
>> You can see that the overloads are working as expected in the tests:
>>
>> https://github.com/person142/numba_special/blob/master/numba_special/tests/test_functions.py
>>
>> (Note that the development version of Numba is required because of
>> this issue: https://github.com/numba/numba/issues/4133)
>>
>> Of course the pointers/overloads that you're creating are highly
>> structured, so you can just generate the necessary glue code:
>>
>> https://github.com/person142/numba_special/blob/master/generate_overloads.py
>>
>> though it only handles simple function signatures right now. All in
>> all I think it works pretty well-you only use 100% officially
>> supported pieces of all the libraries involved. I'll probably work on
>> extending it more in the next couple of days.
>
>
> Very nice, thanks Josh. That's all pretty clean and understandable code.
>
> Cheers,
> Ralf
>
>>
>> On Thu, Jun 20, 2019 at 5:31 PM Nathaniel Smith <njs at pobox.com> wrote:
>> >
>> > On Thu, Jun 20, 2019 at 1:41 PM Stanley Seibert <sseibert at anaconda.com> wrote:
>> > >
>> > > Well, we'd prefer not to have an unnecessary indirection (and the overhead of making a nested call to the scalar inside the length 1 loop) since the user is doing their own iteration in Numba, but if that's the only option, we can certainly work around it.
>> >
>> > Well, it's computers, so there's never just one option :-). But if you
>> > can consume existing ufuncs and get 95% of the performance, then it
>> > might not be worth building custom infrastructure to expose
>> > scipy.special's raw scalar functions + an ad hoc metadata format + ad
>> > hoc tools to consume the metadata. Intuitively it seems like a few
>> > extra integer operations at entry and exit might not be noticeable at
>> > all compared to the cost of computing a bessel function or whatever,
>> > but I haven't checked.
>> >
>> > -n
>> >
>> > --
>> > Nathaniel J. Smith -- https://vorpus.org
>> > _______________________________________________
>> > SciPy-Dev mailing list
>> > SciPy-Dev at python.org
>> > https://mail.python.org/mailman/listinfo/scipy-dev
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev