[Numpy-discussion] Starting to work on runtime plugin system for plugin (automatic sse optimization, etc...)

[David Cournapeau]

On Tue, Apr 29, 2008 at 1:00 AM, Stéfan van der Walt <stefan at sun.ac.za> wrote:

>  I assume that, since you call it a plugin system, it can be done at
>  runtime a-la ctypes?

I am not sure to understand what you mean exactly by a-la ctypes, but
yes, the actual implementation of the npyw_* functions would be
decided at runtime, that's the whole point. For example, instead of
using directly cblas_*dot* functions in blasdot, we would use
npyw_cblas*dot* functions, which would point to something in SSE3
optimized atlas if run on SSE3 cpu, etc... For the actual code change
in numpy and scipy to be minimal, it should only involve renaming
functions, which is what this first prototype focus on.

Once the sytem is ready to be integrated (not anytime soon; for once,
we would need support for the build system to build dynamically loaded
libraries), this would mean numpy and scipy would work like matlab,
which ship with different blas/lapack (atlas_sse, atlas_sse2, mkl),
without the user to have to deal with this kind of low level details.

It would also help to make (if only by making the incentive) a cleaner
difference between pure C implementation and python C api boilerplate
code. I think that optimizing some ufuncs with SSE and co without a
runtime optimization would be a nightmare to deploy otherwise. It is
basically linked to the directions I am the most interested in for
future python numpy releases

cheers,

David