[Numpy-discussion] Openmp support (was numpy's future (1.1 and beyond): which direction(s) ?)

[Gnata Xavier]

Matthieu Brucher wrote:
>
>     It is a real problem in some communities like astronomers and images
>     processing people but the lack of documentation is  the first one,
>     that
>     is true.
>
>
> Even in those communities, I think that a lot could be done at a 
> higher level, as what IPython1 does (tasks parallelism).
>
> Matthieu

Well it is not that easy. We have several numpy code following like this :
1) open an large data file to get a numpy array
2) perform computations on this array (I'm only talking of the numpy 
part here. scipy is something else)
3) Write the result is another large file

It is so simple to write using numpy :)
Now, if I want to have several exe, step 3 is often a problem. The only 
simple way to speed this up is to slit step 2 into threads (assuming 
that there is no other possible optimisation like sse which is false but 
out of the scope of numpy users).

Using C, we can do that using OpenMP pragma. It may not be optimal but 
it radio speedup/time_to_code is very large :) Now, we are switching 
from C to numpy because we cannot spend that much time to play with 
gdb/pointers to open an image anymore.


Xavier
ps : I have seen your blog and you can send me an email off line about 
this topic and what you are doing :)