[SciPy-Dev] Numba as a dependency for SciPy?

[Pauli Virtanen]

Hi,

Ralf Gommers kirjoitti 06.03.2018 klo 05:06:
> Goal of this email: start a discussion to decide whether we'd be okay with
> relying on Numba as a dependency, now or in 1-2 years' time.

I think the main concerns indeed are portability and maturity.

The advantages of Numba on the other hand are clear, as it mostly avoids 
the multiple-language mess.

We might also want to consider using cffi at some point. Currently, we 
basically just have Cython, f2py, and hand-written C code for ffi.

I guess LLVM supports most of the architectures we would be interested 
in, but e.g. the fact that ARM does not work yet is not so nice. 
Moreover, libLLVM is 50+ megabyte blob, but maybe today when people run 
text editors on web browsers instead of vice versa that's not a big deal.

The idea that we could use `scipy._lib.jit` that's either a noop or 
Numba jit does not sound good in practice: for code where the JIT is 
wanted, the performance without it is likely unacceptable. (For PyPy, 
the no-op decorator in principle could be acceptable, but only with 
numpypy which IIUC is not production ready currently, cpyext+numpy 
likely won't be faster than CPython.) Moreover, we presumably would like 
to use features such as `numba.cfunc`. So as I see it, either Numba is a 
hard dependency, or we don't use it.

On maturity: I don't know what is the API stability status for Numba, 
presumably the basic API is stable.

Numba debugging also in my experience has several paper cuts, e.g., the 
compilation and runtime errors are cryptic --- they assume you know how 
numba works, and don't include such niceties as line numbers or useful 
tracebacks, etc. Maybe this will improve in coming years.

	Pauli