[SciPy-Dev] Numba as a dependency for SciPy?

[Matthew Brett]

On Sat, Mar 10, 2018 at 5:34 AM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
>
> On Thu, Mar 8, 2018 at 2:00 AM, Matthew Brett <matthew.brett at gmail.com>
> wrote:
>>
>> Hi,
>>
>> On Thu, Mar 8, 2018 at 9:38 AM, Pauli Virtanen <pav at iki.fi> wrote:
>> > Ralf Gommers kirjoitti 08.03.2018 klo 08:04:
>> > [clip]
>> >>
>> >> Also, I don't think performance will necessarily be unacceptable. There
>> >> are
>> >> a bunch of places in the existing code base where we can throw in @jit
>> >> and
>> >> get speedups basically for free. Performance in the noop case will then
>> >> be
>> >> what it is today - not great, but apparently also not enough of a
>> >> problem
>> >> that someone has attempted to go to Cython.
>> >
>> >
>> > I guess you agree that Numba would regardless be declared a dependency
>> > in
>> > setup.py? People on unsupported arches can edit it away manually.
>> >
>> > For computational tight loops operating on arrays when Numba is used as
>> > an
>> > alternative to Cython/C/Fortran, there probably will be a performance
>> > hit in
>> > the ballpark of 100x.
>> >
>> > If we are planning to use numba features more fully, e.g. numba.cfunc
>> > e.g.
>> > to write callback functions, that would also require Numba as a hard
>> > dependency.
>>
>> If we were at the top of the stack, like pystatsmodels, then this
>> would be reasonable, but, if we make numba a dependency, that makes
>> numba a dependency for almost anyone doing scientific computing.  I
>> think we do have to care about people not running on Intel.   If we
>> make numba an optional dependency, it gives us an additional
>> maintenance burden, because we'd have to check for each numba segment,
>> whether it is going to be disabling for a user without numba.
>>
>> Is there anything we have at the moment where Cython won't get us into
>> the ballpark?
>
>
> That's pretty much an irrelevant question - Cython is typically 2x to 4x
> slower so that's in the ballpark, however the problem is ease of use with
> Cython is so much worse that a lot of code simply won't get ported and just
> stays pure Python.

I must say, I haven't had that experience myself.  Yes, the barrier to
entry is relatively high, but once you're over that, I have found
Cython fairly agreeable to work with.

Cheers,

Matthew