[SciPy-Dev] Numba as a dependency for SciPy?

[Ralf Gommers]

On Thu, Mar 8, 2018 at 2:00 AM, Matthew Brett <matthew.brett at gmail.com>
wrote:

> Hi,
>
> On Thu, Mar 8, 2018 at 9:38 AM, Pauli Virtanen <pav at iki.fi> wrote:
> > Ralf Gommers kirjoitti 08.03.2018 klo 08:04:
> > [clip]
> >>
> >> Also, I don't think performance will necessarily be unacceptable. There
> >> are
> >> a bunch of places in the existing code base where we can throw in @jit
> and
> >> get speedups basically for free. Performance in the noop case will then
> be
> >> what it is today - not great, but apparently also not enough of a
> problem
> >> that someone has attempted to go to Cython.
> >
> >
> > I guess you agree that Numba would regardless be declared a dependency in
> > setup.py? People on unsupported arches can edit it away manually.
> >
> > For computational tight loops operating on arrays when Numba is used as
> an
> > alternative to Cython/C/Fortran, there probably will be a performance
> hit in
> > the ballpark of 100x.
> >
> > If we are planning to use numba features more fully, e.g. numba.cfunc
> e.g.
> > to write callback functions, that would also require Numba as a hard
> > dependency.
>
> If we were at the top of the stack, like pystatsmodels, then this
> would be reasonable, but, if we make numba a dependency, that makes
> numba a dependency for almost anyone doing scientific computing.  I
> think we do have to care about people not running on Intel.   If we
> make numba an optional dependency, it gives us an additional
> maintenance burden, because we'd have to check for each numba segment,
> whether it is going to be disabling for a user without numba.
>
> Is there anything we have at the moment where Cython won't get us into
> the ballpark?


That's pretty much an irrelevant question - Cython is typically 2x to 4x
slower so that's in the ballpark, however the problem is ease of use with
Cython is so much worse that a lot of code simply won't get ported and just
stays pure Python.


>   If not, my preference would be to wait for a year or
> so, to see how things turn out.
>

Yes, agreed. The consensus seems to be "no" for now - we can revisit once
ARM & co are better supported.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180309/9a4bd4b0/attachment.html>