[Numpy-discussion] Cython-based OpenMP-accelerated quartic polynomial solver

[Nathan Goldbaum]

On Wed, Sep 30, 2015 at 3:20 AM, Juha Jeronen <juha.jeronen at jyu.fi> wrote:

> Hi,
>
> On 30.09.2015 03:48, Chris Barker - NOAA Federal wrote:
>
>> This sounds pretty cool -- and I've had a use case. So it would be
>> nice to get into Numpy.
>>
>> But: I doubt we'd want OpenMP dependence in Numpy itself.
>>
>
> It is indeed a good point not to add new dependencies for a small feature
> such as this. From a software engineering point of view, I fully agree.
>
> Note however that with the current version of the code, not having OpenMP
> will severely limit the performance, especially on quad-core machines,
> since an important factor in the speed comes from the parallel processing
> of the independent problem instances.
>
> But I suppose there may be another technical option to support multiple
> cores (doesn't NumPy already do this in some of its routines?). OpenMP was
> just the most convenient solution from the implementation viewpoint.
>
>
> But maybe a pure Cython non-MP version?
>>
>
> That is of course possible. IIRC, changing the processing to occur on a
> single core should only require replacing Cython.parallel.prange() with
> range() in the array processing loops.
>
> (Note range(), not xrange(), even for Python 2.x - Cython compiles a loop
> using range() into an efficient C loop if some simple compile-time
> conditions are fulfilled.)
>
>
> Are we putting Cuthon in Numpy now? I lost track.
>>
>
> I thought so? But then again, I'm just a user :)


For what it's worth (no idea what the order of magnitude of the technical
risk is for something like this in a library like numpy), it's actually
quite simple to dynamically test for OpenMP support at install time.

Here's what we do in yt:

https://bitbucket.org/yt_analysis/yt/src/f8934e13abaf349f58c52c076320d44ee4f61a7f/yt/utilities/lib/setup.py?at=yt&fileviewer=file-view-default#setup.py-8

Basically, just try to compile a simple OpenMP test program in a
subprocess. If that succeeds, then great, we can add -fopenmp as a
compilation flag. If not, don't do that.

This was implemented when Apple switched from GCC 4.2 to LLVM/Clang in OS X
10.7. We've had exactly one bug report about this, and it wasn't even
related to OpenMP, instead it was an issue with the way we were using
subprocess which broke when someone had set "CC = ccache gcc".

Hope that's helpful,

Nathan Goldbaum


>
>
>
>  -J
>
> -------------------------------------------------
> Juha Jeronen, Ph.D.
> juha.jeronen at jyu.fi
> University of Jyväskylä
> Department of Mathematical Information Technology
> -------------------------------------------------
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20150930/b7f4d946/attachment.html>