[Cython] [Re] OpenMP thread private variable not recognized (bug report + discussion)

Leon Bottou leon at bottou.org
Wed Aug 13 20:21:02 CEST 2014


> > I am making heavy uses of OpenBlas which also uses OpenMP.
> > Using the same queue manager prevents lots of CPU provisioning problem.
> > Using multiple queue managers in the same code does not work as well
> > because they are not aware of what the other one is doing.
> 
> Normally OpenBLAS is built without OpenMP. Also, OpenMP is not fork safe
> (cf. multiprocessing) but OpenBLAS' own threadpool is. So it is
recommended
> to build OpenBLAS without OpenMP dependency.
> 
> That is: If you build OpenBLAS with OpenMP, numpy.dot will hang if used
> together with multiprocessing.

I am effectively using a version of openblas built with openmp because
Debian used to compile openblas this way. They seem to have reverted now.
Note than I cannot use python multiprocessing because my threads work on a
very large state vector.  My current solution is to use python threading and
nogil cython compiled routines but this sometimes lead to weird effects
provisioning threads. 

This is why I wanted to try the pure openmp solution and found the
aforementioned bug in cython.parallel.

Is there somebody actively trying to make cython.parallel work correctly?
- If yes, then my bug report should be of interest to this person.
- If no, then one should avoid (and possibly deprecate) cython.parallel and
find other ways to do things. 

Thanks to the replies.

- L.




More information about the cython-devel mailing list