On Wed, Sep 30, 2015 at 3:20 AM, Juha Jeronen <juha.jeronen@jyu.fi> wrote:

Hi,

On 30.09.2015 03:48, Chris Barker - NOAA Federal wrote:

This sounds pretty cool -- and I've had a use case. So it would be
nice to get into Numpy.

But: I doubt we'd want OpenMP dependence in Numpy itself.

It is indeed a good point not to add new dependencies for a small feature such as this. From a software engineering point of view, I fully agree.

Note however that with the current version of the code, not having OpenMP will severely limit the performance, especially on quad-core machines, since an important factor in the speed comes from the parallel processing of the independent problem instances.

But I suppose there may be another technical option to support multiple cores (doesn't NumPy already do this in some of its routines?). OpenMP was just the most convenient solution from the implementation viewpoint.

But maybe a pure Cython non-MP version?

That is of course possible. IIRC, changing the processing to occur on a single core should only require replacing Cython.parallel.prange() with range() in the array processing loops.

(Note range(), not xrange(), even for Python 2.x - Cython compiles a loop using range() into an efficient C loop if some simple compile-time conditions are fulfilled.)

Are we putting Cuthon in Numpy now? I lost track.

I thought so? But then again, I'm just a user :)

For what it's worth (no idea what the order of magnitude of the technical risk is for something like this in a library like numpy), it's actually quite simple to dynamically test for OpenMP support at install time.

Here's what we do in yt:

https://bitbucket.org/yt_analysis/yt/src/f8934e13abaf349f58c52c076320d44ee4f61a7f/yt/utilities/lib/setup.py?at=yt&fileviewer=file-view-default#setup.py-8

Basically, just try to compile a simple OpenMP test program in a subprocess. If that succeeds, then great, we can add -fopenmp as a compilation flag. If not, don't do that.

This was implemented when Apple switched from GCC 4.2 to LLVM/Clang in OS X 10.7. We've had exactly one bug report about this, and it wasn't even related to OpenMP, instead it was an issue with the way we were using subprocess which broke when someone had set "CC = ccache gcc".

Hope that's helpful,

Nathan Goldbaum

-J

-------------------------------------------------
Juha Jeronen, Ph.D.
juha.jeronen@jyu.fi
University of Jyväskylä
Department of Mathematical Information Technology
-------------------------------------------------

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion