[Numpy-discussion] Proposed Roadmap Overview

[Christopher Jordan-Squire]

On Mon, Feb 20, 2012 at 9:18 AM, Dag Sverre Seljebotn
<d.s.seljebotn at astro.uio.no> wrote:
> On 02/20/2012 08:55 AM, Sturla Molden wrote:
>> Den 20.02.2012 17:42, skrev Sturla Molden:
>>> There are still other options than C or C++ that are worth considering.
>>> One would be to write NumPy in Python. E.g. we could use LLVM as a
>>> JIT-compiler and produce the performance critical code we need on the fly.
>>>
>>>
>>
>> LLVM and its C/C++ frontend Clang are BSD licenced. It compiles faster
>> than GCC and often produces better machine code. They can therefore be
>> used inside an array library. It would give a faster NumPy, and we could
>> keep most of it in Python.
>
> I think it is moot to focus on improving NumPy performance as long as in
> practice all NumPy operations are memory bound due to the need to take a
> trip through system memory for almost any operation. C/C++ is simply
> "good enough". JIT is when you're chasing a 2x improvement or so, but
> today NumPy can be 10-20x slower than a Cython loop.
>

I don't follow this. Could you expand a bit more? (Specifically, I
wasn't aware that numpy could be 10-20x slower than a cython loop, if
we're talking about the base numpy library--so core operations. I'm
also not totally sure why a JIT is a 2x improvement or so vs. cython.
Not that a disagree on either of these points, I'd just like a bit
more detail.)

Thanks,
Chris

> You need at least a slightly different Python API to get anywhere, so
> numexpr/Theano is the right place to work on an implementation of this
> idea. Of course it would be nice if numexpr/Theano offered something as
> convenient as
>
> with lazy:
>     arr = A + B + C # with all of these NumPy arrays
> # compute upon exiting...
>
> Dag
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion