Re: [Numpy-discussion] NumPy-Discussion OpenBLAS and dotblas
Hi Nathaniel. Thanks for your prompt reply. I think numpy is a wonderful project, and you all do a great job moving it forward. If you ask what would my vision for maturing numpy, I would like to see a grouping of linalg matrix-operation functionality into a python level package, exactly the opposite of more tightly tying linalg into the core of numpy. The orthagonality would allow goups like PyOpenCL to reuse the matrix operations on data located off the CPU's RAM, just to give one example; and make it easier for non-numpy developers to create a complete replacement of lapack with other implementations. Much of the linalg package would of course be implemented in c or fortran, but the interface to ndarray would use the well-established idea of contiguous matrices with shapes, strides, and a single memory store, supporting only numeric number types. I suggested cffi since it provides a convienent and efficient interface to ndarray. Thus python could remain as a thin wrapper over the calls out to c-based libraries much like lapack_lite does today, but at the python level rather that the capi level. Yes, a python-based interface would slows the code down a bit, but I would argue that 1. the current state of lapack_litemodule.c and umath_linalg.c.src, with its myriad of compile-time macros and complex code paths, scares people away from contributing to the ongoing maintenance of the library while tying the code very closely to the lapack routines, and 2. matrices larger than 3x3 or so should be spending most of the computation time in the underlying lapack/blas library irregardless of whether the interface is python-based or capi-based. Matti On 10/08/2014 8:00 PM, numpy-discussion-request@scipy.org wrote:
Date: Sat, 9 Aug 2014 21:11:19 +0100 From: Nathaniel Smith
Subject: Re: [Numpy-discussion] NumPy-Discussion OpenBLAS and dotblas To: Discussion of Numerical Python On Sat, Aug 9, 2014 at 8:35 PM, Matti Picus
wrote: Hi. I am working on numpy in pypy. It would be much more challenging for me if you merged more code into the core of numpy, Hi Matti,
I can definitely see how numpy changes cause trouble for you, and sympathize. But, can you elaborate on what kind of changes would make your life easier *that also* help make numpy proper better in their own right? Because unfortunately, I don't see how we can reasonably pass up on improvements to numpy if the only justification is to make numpypy's life easier. (I'd also love to see pypy become usable for general numerical work, but not only is it not there now, I don't see how numpypy will ultimately get us there even if we do help it along -- almost none of the ecosystem can get by numpy's python-level APIs alone.) But obviously if there are changes that are mutually beneficial, well then, that's a lot easier to justify :-)
-n
participants (7)
-
Charles R Harris
-
cjw
-
Matti Picus
-
Nathaniel Smith
-
Ralf Gommers
-
Robert Kern
-
Sturla Molden