[Numpy-discussion] SSEPlus + Framewave

[David Cournapeau]

On Wed, Aug 13, 2008 at 3:28 AM, Holger Rapp <Rapp at mrt.uka.de> wrote:

> What do you mean by compiling incompatible? It is my understanding
> that (for example) Framewave (but also IPP) come in different flavors
> (32bit, 64bit) which of course must be compiled in at compile time.
> But which CPU is available and which features it delivers is indeed
> done at runtime (framewave: fwStaticInit()), the choice of how to
> implement things with which assembler code is then up to the framewave
> library.

Yes, but not all libraries do that. Typically, Atlas doesn't.

> I do not consider it a good idea to write a own dispatcher library
> into numpy to choose which opcode to use.

Having a clear separation between computational part and "boilerplate"
has another big advantage: to make it easier to provide numpy
facilities other C extensions (typically, I would like to be able to
use special functions, fft, blas/lapack, etc... at the C level in some
scipy extensions). It also makes testing, benchmarking easier.

This is cleaner, too. And once you have this separation, having a
plugin system is not that difficult.

> Or do it get you completly wrong? Is your intention to make a plugin
> architecture in the sense of: copy some directory with libs and config
> in your site-packages and then your multiplications are much faster?

Not exactly: for binaries, we would compile all architectures, and it
will be transparant to the user. The plugin-system would be an
internal thing, the user would not be aware of it. Like matlab does it
for example.

> would consider such a framework a bit overengineered, since speedy
> calculations are a nice feature for every numpy user.

But that's the point: having optimization for every user, without the
user having to care about which numpy to install.

cheers,

David