[Numpy-discussion] Fast threading solution thoughts

[Dag Sverre Seljebotn]

Brian Granger wrote:
> Hi,
>
> This is relevant for anyone who would like to speed up array based
> codes using threads.
>
> I have a simple loop that I have implemented using Cython:
>
> def backstep(np.ndarray opti, np.ndarray optf,
>              int istart, int iend, double p, double q):
>     cdef int j
>     cdef double *pi
>     cdef double *pf
>     pi = <double *>opti.data
>     pf = <double *>optf.data
>
>     with nogil:
>         for j in range(istart, iend):
>             pf[j] = (p*pi[j+1] + q*pi[j])
>
> I need to call this function *many* times and each time cannot be
> performed until the previous time is completely as there are data
> dependencies.  But, I still want to parallelize a single call to this
> function across multiple cores (notice that I am releasing the GIL
> before I do the heavy lifting).
>   
A quick digression:

It would be interesting to see how a spec would look for integrating 
OpenMP natively into Cython for these kinds of purposes. Cython is still 
flexible as a language after all. Avoiding language bloat is also 
important, but it is difficult to know what kind of balance can be 
struck before some kind of spec is worked out. Has anyone managed to use 
OpenMP with Cython code in a nice way already?

I couldn't do any work on it right now but it could sit in the pipeline 
for the year to come. Also I have a strong feeling that making the spec 
and language design issues would take more time than the actual 
implementation of it anyway.

Dag Sverre