[SciPy-Dev] Numba as a dependency for SciPy?

[Joshua Wilson]

> if someone is interested in trying to implement some SciPy functions with Numba implementations

When this thread started I actually did just that for a small part of
scipy.special. Code is here:

https://github.com/person142/special

It currently implements `special.loggamma` and some private functions
in special (sinpi, cospi, polynomial evaluation) that are needed to
support it. I'm currently seeing a factor of 2 slowdown and trying to
figure out why, but I'm very interested in figuring out how to close
the speed gap/porting more functions.

- Josh

On Thu, Mar 8, 2018 at 8:44 AM, Stanley Seibert <sseibert at anaconda.com> wrote:
> TBH, I agree with this general sentiment.  This thread has been very
> valuable to clarify the Numba team's understanding of what the SciPy
> community needs from a compilation solution.  Our trajectory is good, but
> we're not quite there yet for a project that needs to be as conservative
> about dependencies as SciPy.  We will keep working to get there, though.
>
> However, if someone is interested in trying to implement some SciPy
> functions with Numba implementations, there's nothing blocking that work in
> a separate repository as an experiment.  (One of the Numba developers has
> already named this hypothetical project "Scumba," which I quite like.)  If
> anyone does decide to try this, please make sure to ping the Numba
> developers on Gitter.  We will learn a great deal from the effort, I think.
>
> On Thu, Mar 8, 2018 at 4:00 AM, Matthew Brett <matthew.brett at gmail.com>
> wrote:
>>
>> Hi,
>>
>> On Thu, Mar 8, 2018 at 9:38 AM, Pauli Virtanen <pav at iki.fi> wrote:
>> > Ralf Gommers kirjoitti 08.03.2018 klo 08:04:
>> > [clip]
>> >>
>> >> Also, I don't think performance will necessarily be unacceptable. There
>> >> are
>> >> a bunch of places in the existing code base where we can throw in @jit
>> >> and
>> >> get speedups basically for free. Performance in the noop case will then
>> >> be
>> >> what it is today - not great, but apparently also not enough of a
>> >> problem
>> >> that someone has attempted to go to Cython.
>> >
>> >
>> > I guess you agree that Numba would regardless be declared a dependency
>> > in
>> > setup.py? People on unsupported arches can edit it away manually.
>> >
>> > For computational tight loops operating on arrays when Numba is used as
>> > an
>> > alternative to Cython/C/Fortran, there probably will be a performance
>> > hit in
>> > the ballpark of 100x.
>> >
>> > If we are planning to use numba features more fully, e.g. numba.cfunc
>> > e.g.
>> > to write callback functions, that would also require Numba as a hard
>> > dependency.
>>
>> If we were at the top of the stack, like pystatsmodels, then this
>> would be reasonable, but, if we make numba a dependency, that makes
>> numba a dependency for almost anyone doing scientific computing.  I
>> think we do have to care about people not running on Intel.   If we
>> make numba an optional dependency, it gives us an additional
>> maintenance burden, because we'd have to check for each numba segment,
>> whether it is going to be disabling for a user without numba.
>>
>> Is there anything we have at the moment where Cython won't get us into
>> the ballpark?  If not, my preference would be to wait for a year or
>> so, to see how things turn out.
>>
>> Cheers,
>>
>> Matthew
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>