[SciPy-Dev] Numba as a dependency for SciPy?

[CJ Carey]

I think adding a required runtime dependency may be overly restrictive,
given scipy's position near(-ish) the base of the scientific computing
pyramid.

Would it be possible to run numba-optimized code on systems with numba
installed without impacting "vanilla" users?


On Tue, Mar 6, 2018 at 3:03 PM Tyler Reddy <tyler.je.reddy at gmail.com> wrote:

> Interesting discussion. Would our plan be to support both side-by-side for
> a while & just see what happens with the evolution of the ecosystem? If
> there's no clear winner in the short-term would we discourage PRs that
> simply migrate from Cython to numba for say 1.5 x performance increase?
> What about an algorithm that mixes the two approaches -- some numba and
> some Cython components for whatever reason -- is that discouraged?
>
> It looks like numba plays ok with airspeed velocity -- presumably mixing
> Cython / numba in our suite will be ok?
>
>
>
> On 5 March 2018 at 21:06, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
>> Hi all,
>>
>> Goal of this email: start a discussion to decide whether we'd be okay
>> with relying on Numba as a dependency, now or in 1-2 years' time.
>>
>> Context: in https://github.com/pydata/sparse/issues/126 a discussion is
>> ongoing about whether to adopt Cython or Numba, with Numba being preferred
>> by the majority. That `sparse` package is meant to provide sparse *arrays*
>> that down the line should either be replacing our current sparse *matrices*
>> or at least be integrated in scipy.sparse in addition to them. See
>> https://github.com/scipy/scipy/issues/8162 and
>> https://github.com/hameerabbasi/sparse-ndarray-protocols for more
>> details on that.
>>
>> Also related is the question from Serge Guelton some weeks ago about
>> whether we'd want to rely on Pythran:
>> https://mail.python.org/pipermail/scipy-dev/2018-January/022325.html
>>
>> On that Pythran thread I commented that we'd want to take these aspects
>> into account:
>> - portability
>> - performance
>> - maturity
>> - maintenance status (active devs, how quick do bugs get fixed after a
>> release with an issue)
>> - ease of use (@jit vs. Pythran comments vs. translate to .pyx syntax)
>> - size of generated binaries
>> - templating support for multiple dtypes
>> - debugging and optimization experience/tool
>>
>> Debugging is one of the ones where I'd say Numba is still worse than
>> Cython, however that's being resolved as we speak:
>> https://github.com/numba/numba/issues/2788
>>
>> One thing I missed in the above list is dependencies: while our use of
>> Cython only adds a build-time dependency, Numba would add a run-time
>> dependency. Given that binary wheels and conda packages for all major
>> platforms are available that's not a showstopper, but it matters.
>>
>> Overall I'd say that:
>> - Numba is better than Cython at: performance, ease of use, size of
>> generated binaries, and templating support for multiple dtypes. Possibly
>> also maintenance status right now.
>> - Numba and Cython are about equally good at portability (I think, not
>> much data about exotic platforms for Numba).
>> - Cython is better than Numba at: maturity, debugging (but not for long
>> anymore probably), dependencies.
>>
>> I'm usually pretty conservative in these things, but considering the
>> above I'm leaning towards saying use of Numba should be allowed in the
>> future. The added run-time dependency is the one major downside that's
>> going to stay, however compared to our Fortran headaches that's a
>> relatively small issue.
>>
>> Thoughts?
>>
>> Cheers,
>> Ralf
>>
>>
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180306/7226824a/attachment-0001.html>