[SciPy-Dev] Numba as a dependency for SciPy?

[Eleftherios Garyfallidis]

I looked recently quite deeply into Numba as a possible dependency for
DIPY. My opinion is that Numba is not ready to be a dependency for Dipy and
neither numpy/scipy too. Actually, you can often easily beat Numba's speed
with a bit of Cython code. That is because often Numba is not smart enough
in porting some code from Python to C. I think the benchmarks out there are
a bit misleading by comparing only very specific things or very simple
functions. I understand the excitement but we should be realistic too. The
way you write fast code in C can be very different from how you write in
Python. Definitely Numba has incredible potential but we need to give it a
bit more time to mature also we need more generic benchmarks. I would love
to hear what the creators of Numba think too.

Best,
Eleftherios


On Sat, Mar 10, 2018 at 6:07 PM Matthew Brett <matthew.brett at gmail.com>
wrote:

> On Sat, Mar 10, 2018 at 5:34 AM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
> >
> >
> > On Thu, Mar 8, 2018 at 2:00 AM, Matthew Brett <matthew.brett at gmail.com>
> > wrote:
> >>
> >> Hi,
> >>
> >> On Thu, Mar 8, 2018 at 9:38 AM, Pauli Virtanen <pav at iki.fi> wrote:
> >> > Ralf Gommers kirjoitti 08.03.2018 klo 08:04:
> >> > [clip]
> >> >>
> >> >> Also, I don't think performance will necessarily be unacceptable.
> There
> >> >> are
> >> >> a bunch of places in the existing code base where we can throw in
> @jit
> >> >> and
> >> >> get speedups basically for free. Performance in the noop case will
> then
> >> >> be
> >> >> what it is today - not great, but apparently also not enough of a
> >> >> problem
> >> >> that someone has attempted to go to Cython.
> >> >
> >> >
> >> > I guess you agree that Numba would regardless be declared a dependency
> >> > in
> >> > setup.py? People on unsupported arches can edit it away manually.
> >> >
> >> > For computational tight loops operating on arrays when Numba is used
> as
> >> > an
> >> > alternative to Cython/C/Fortran, there probably will be a performance
> >> > hit in
> >> > the ballpark of 100x.
> >> >
> >> > If we are planning to use numba features more fully, e.g. numba.cfunc
> >> > e.g.
> >> > to write callback functions, that would also require Numba as a hard
> >> > dependency.
> >>
> >> If we were at the top of the stack, like pystatsmodels, then this
> >> would be reasonable, but, if we make numba a dependency, that makes
> >> numba a dependency for almost anyone doing scientific computing.  I
> >> think we do have to care about people not running on Intel.   If we
> >> make numba an optional dependency, it gives us an additional
> >> maintenance burden, because we'd have to check for each numba segment,
> >> whether it is going to be disabling for a user without numba.
> >>
> >> Is there anything we have at the moment where Cython won't get us into
> >> the ballpark?
> >
> >
> > That's pretty much an irrelevant question - Cython is typically 2x to 4x
> > slower so that's in the ballpark, however the problem is ease of use with
> > Cython is so much worse that a lot of code simply won't get ported and
> just
> > stays pure Python.
>
> I must say, I haven't had that experience myself.  Yes, the barrier to
> entry is relatively high, but once you're over that, I have found
> Cython fairly agreeable to work with.
>
> Cheers,
>
> Matthew
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180311/3b880be2/attachment.html>