[Numpy-discussion] Psyco MA?

[Joachim Saul]

* Tim Churches [2003-02-08 01:25]:
> On Sat, 2003-02-08 at 10:57, Joachim Saul wrote:
> > * Tim Churches [2003-02-08 00:07]:
> > > However, I agree 100% about the potential for leveraging Pyrex in
> > > Numarray. Not just in Numarray, but around it, too. The Numarray team
> > > should open serious talks with Greg Ewing about Numarray-enabling Pyrex.
> >
> > What is it that needs to be "enabled"? Pyrex handles Numeric (see
> > Pyrex FAQ), why should it not handle Numarray? AFAIK Pyrex
> > contains no code to specifically support Numeric, and it should
> > therefore be straightforward to use it with Numarray as well.
>
> Hmmm, maybe re-implementing MA in Pyrex is possible right now. Double
> hmmm....

Please check out the Pyrex doc. It's actually very easy right now,
*if* you can live without "sequence operators" such as slicing,
list comprehensions... but this is going to be supported, again
according to the doc.

Here is a exerpt from an extension module that I have built using
Pyrex and Numeric, following the instructions in the Pyrex-FAQ:

cdef extern int decomp(int, double*, double*, double, double, double)

cdef extern from "Numeric/arrayobject.h":
    struct PyArray_Descr:
        int type_num, elsize
        char type
    ctypedef class PyArrayObject [type PyArray_Type]:
        cdef char *data
        cdef int nd
        cdef int *dimensions,*strides
        cdef PyArray_Descr *descr
    object PyArray_FromDims(int, int*, int)
    void import_array()

def _decomp(PyArrayObject z_arr, PyArrayObject r_arr,
            double p, double vs, double sigma):
    cdef double *z, *r
    cdef int n

    n = z_arr.dimensions[0]
    z, r = <double*> z_arr.data, <double*> r_arr.data
    decomp(n, z, r, p, vs, sigma)

This is rather crude code that doesn't check for the type of the
arrays nor their dimension, but it does what I want right now and
if I find the time I'll certainly make it more general. Those
checks are actually performed in yet another Python layer.

As you can see, the above looks like "strongly typed" Python. From
a C-programmers perspective, if find this is extremely cool. If one
leaves the type out, then the argument can be any Python object.

What I like about Pyrex is that you can mix Python and C calls at
your convenience. For example, I may call (C-like)

    arr = PyArray_FromDims(1, &n, PyArray_DOUBLE)

but could have also used a corresponding Python construct like

    from Numeric import zeros
    arr = zeros(n, 'd')

I expect the latter to be slower (not tested), but one can take
Python code "as is" and "compile" it using Pyrex. This already
increases performance and one can then conveniently replace as
much Python code as needed with the corresponding C functions,
which (presumably) will again speed up the code significantly.
The bottle necks are finally moved to external C files and
treated like a C library.

Cheers,
Joachim