On Sun, 2005-04-17 at 23:43, Simon Burton wrote:

On Fri, 15 Apr 2005 06:44:02 -0400 Todd Miller <jmiller@stsci.edu> wrote:

...

On Fri, 2005-04-15 at 16:04 +1000, Simon Burton wrote:

...

Hi,

I see there is a cholesky_decomposition routine in numarray, but we are also needing the corresponding cholesky solver. Is this in the pipeline,

No. Most of the add-on subpackages in numarray, with the exception of convolve, image, and nd_image, are ports from Numeric.

Ok, thanks Todd; we will have a go at porting this solver then. If you have any more advice on how to get started with this that would be much appreciated.

If you're doing a port of something that already works for Numeric chances are good that numarray's Numeric compatibility API will make things "just work." In any case, be sure to use the compatibility API since it's the easiest path forward to Numeric3 should that effort prove successful (which I think it will). Usually what's involved in porting from Numeric to numarray is just making sure that the numarray files can be used rather than the Numeric header files. I think the style we used for matplotlib, while not fully general, is the simplest and best compromise: #ifdef NUMARRAY #include "numarray/arrayobject.h" #else #include "Numeric/arrayobject.h" #endif In setup.py, you have to pass extra_compile_args=["-DNUMARRAY=1"] or similar to the Extension() constructions to build for numarray. There are more details we could discuss if you want to build for both Numeric and numarray simultaneously. Two limitations of the numarray Numeric compatible C-API are: (1) a partially compatible array descriptor structure (PyArray_Descr) and (2) the UFunc C-API. Generally, neither of those is an issue, but for large projects (e.g. scipy) they matter. Good luck porting. Feel free to ask questions either on the list or privately if you run into trouble. Regards, Todd

On Apr 15, 2005, at 10:09 AM, Piotr Luszczek wrote:

Hi all,

the Cholesky routine that's been mentioned (dpotrs) is from LAPACK (I apologize if every body knows that).

I'm on the LAPACK team right now and we were wondering if we should provide bindings for Python. It is almost trivial to do with Pyrex. But Numeric and numarray already have some functionality in it. Also, I don't know about popularity of PyLapack.

So my question is if there is a need for the specialized LAPACK routines. And if so, which API it should use (Numeric, numarray, Numeric3, scipy_core, standard array, minimum standard array implementation or array protocol meta info).

Any comments are appreciated,

Piotr Luszczek

If you don't need anything unusual, using the Numeric C-API should be safe. There is the intent to preserve backward compatibility for that in numarray and Numeric3 for the most part (numarray's ufunc api is different however, but it isn't clear you need to use that). Numeric3 and numarray will/do have other capabilities not part of the Numeric api, but again, I suspect that for a first version, one can probably avoid needing those. I'd also like to hear what Travis thinks about this. Perry Greenfield

David M. Cooke wrote:

Piotr Luszczek <luszczek@cs.utk.edu> writes:

...

Hi all,

the Cholesky routine that's been mentioned (dpotrs) is from LAPACK (I apologize if every body knows that).

I'm on the LAPACK team right now and we were wondering if we should provide bindings for Python. It is almost trivial to do with Pyrex. But Numeric and numarray already have some functionality in it. Also, I don't know about popularity of PyLapack.

So my question is if there is a need for the specialized LAPACK routines. And if so, which API it should use (Numeric, numarray, Numeric3, scipy_core, standard array, minimum standard array implementation or array protocol meta info).

You'll probably first want to look at scipy, which already wraps (all? most?) of LAPACK in its scipy.linalg package (including dpotrs :-)

It seems to have almost all routines.

It uses f2py to make the process much easier.

Since you mention you're on the LAPACK team ...

I've been working on redoing the f2c'd LAPACK wrappers in Numeric, updating them to the current version...except: what *is* the current

Current version is 3.0.

version? The patches on netlib are 2-3 years old, and you have to grab

After funding ran out there were only volunteers left. It's hard to get free open-source developers these days.

them separately, file-by-file (can I say how insanely stupid that

Frankly, I had the same comment when I first saw it. Hopefully, next update will straighten things out.

is?). Also ... they break: with some test cases (derived from ones posted to our bug tracker) some routines segfault.

Yes I know. We have postings about it on the mailing list almost weekly.

Is it the LAPACK 3e? If that's the case, we can't use it unless there

LAPACK 3E is only somewhat related to LAPACK. But it's not "current version".

are C versions (Numeric only requires Python and a C compiler; throwing a F90 compiler in there is *not* an option -- we don't even require a F77 compiler).

We've been thinking about languages for a while. CLAPACK user base is too strong to ignore. So we think of keeping F77 as the base language. Or maybe we should do f90toC. f2c and f2j are on Netlib already and f2py has some F90 support.

I ended up using the source from Debian unstable from the lapack3 package, and those work fine.

Again, it's hard to get grant money for support. Thanks for the comments. Piotr

Piotr Luszczek wrote:

Hi all,

the Cholesky routine that's been mentioned (dpotrs) is from LAPACK (I apologize if every body knows that).

I'm on the LAPACK team right now and we were wondering if we should provide bindings for Python. It is almost trivial to do with Pyrex. But Numeric and numarray already have some functionality in it. Also, I don't know about popularity of PyLapack.

Scipy already has extensive bindings for LAPACK. There is even a lot of development that has been done for c-compiled bindings. Right now, scipy_core is being developed to be a single replacement for Numeric/numarray. Lapack bindings are a huge part of that effort. But, as I said, the work has been done (using f2py). The biggest issue is supporting f2c'd versions of Lapack so that folks without Fortran compilers can still install it. scipy_core will allow this. Again, most of the effort is accomplished through f2py and scipy_distutils which are really good tools. Pyrex is nice, but f2py is really, really nice (it even supports wrapping basic c-code).

So my question is if there is a need for the specialized LAPACK routines. And if so, which API it should use (Numeric, numarray, Numeric3, scipy_core, standard array, minimum standard array implementation or array protocol meta info).

I think if LAPACK were going to go through the trouble, it would be best for LAPACK to provide "array protocol" style wrappers. That way any Python array user could take advantage of them. While current scipy users and future scipy_core users do not need LAPACK-provided Python wrappers, we would welcome any native support by the LAPACK team. Again, though, I think this should be done through the array_protocol API. A C-API is likely in the near future as well (which will provide a little speed up for many small arrays). -Travis -Travis