[SciPy-Dev] Numba as a dependency for SciPy?

[Tyler Reddy]

Interesting discussion. Would our plan be to support both side-by-side for
a while & just see what happens with the evolution of the ecosystem? If
there's no clear winner in the short-term would we discourage PRs that
simply migrate from Cython to numba for say 1.5 x performance increase?
What about an algorithm that mixes the two approaches -- some numba and
some Cython components for whatever reason -- is that discouraged?

It looks like numba plays ok with airspeed velocity -- presumably mixing
Cython / numba in our suite will be ok?



On 5 March 2018 at 21:06, Ralf Gommers <ralf.gommers at gmail.com> wrote:

> Hi all,
>
> Goal of this email: start a discussion to decide whether we'd be okay with
> relying on Numba as a dependency, now or in 1-2 years' time.
>
> Context: in https://github.com/pydata/sparse/issues/126 a discussion is
> ongoing about whether to adopt Cython or Numba, with Numba being preferred
> by the majority. That `sparse` package is meant to provide sparse *arrays*
> that down the line should either be replacing our current sparse *matrices*
> or at least be integrated in scipy.sparse in addition to them. See
> https://github.com/scipy/scipy/issues/8162 and https://github.com/
> hameerabbasi/sparse-ndarray-protocols for more details on that.
>
> Also related is the question from Serge Guelton some weeks ago about
> whether we'd want to rely on Pythran: https://mail.python.org/
> pipermail/scipy-dev/2018-January/022325.html
>
> On that Pythran thread I commented that we'd want to take these aspects
> into account:
> - portability
> - performance
> - maturity
> - maintenance status (active devs, how quick do bugs get fixed after a
> release with an issue)
> - ease of use (@jit vs. Pythran comments vs. translate to .pyx syntax)
> - size of generated binaries
> - templating support for multiple dtypes
> - debugging and optimization experience/tool
>
> Debugging is one of the ones where I'd say Numba is still worse than
> Cython, however that's being resolved as we speak:
> https://github.com/numba/numba/issues/2788
>
> One thing I missed in the above list is dependencies: while our use of
> Cython only adds a build-time dependency, Numba would add a run-time
> dependency. Given that binary wheels and conda packages for all major
> platforms are available that's not a showstopper, but it matters.
>
> Overall I'd say that:
> - Numba is better than Cython at: performance, ease of use, size of
> generated binaries, and templating support for multiple dtypes. Possibly
> also maintenance status right now.
> - Numba and Cython are about equally good at portability (I think, not
> much data about exotic platforms for Numba).
> - Cython is better than Numba at: maturity, debugging (but not for long
> anymore probably), dependencies.
>
> I'm usually pretty conservative in these things, but considering the above
> I'm leaning towards saying use of Numba should be allowed in the future.
> The added run-time dependency is the one major downside that's going to
> stay, however compared to our Fortran headaches that's a relatively small
> issue.
>
> Thoughts?
>
> Cheers,
> Ralf
>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180306/81d01b60/attachment.html>