[Cython] prange CEP updated

[Dag Sverre Seljebotn]

On 05/04/2011 12:00 PM, mark florisson wrote:
> On 21 April 2011 20:13, Dag Sverre Seljebotn<d.s.seljebotn at astro.uio.no>  wrote:
>> On 04/21/2011 10:37 AM, Robert Bradshaw wrote:
>>>
>>> On Mon, Apr 18, 2011 at 7:51 AM, mark florisson
>>> <markflorisson88 at gmail.com>    wrote:
>>>>
>>>> On 18 April 2011 16:41, Dag Sverre Seljebotn<d.s.seljebotn at astro.uio.no>
>>>>   wrote:
>>>>>
>>>>> Excellent! Sounds great! (as I won't have my laptop for some days I
>>>>> can't
>>>>> have a look yet but I will later)
>>>>>
>>>>> You're right about (the current) buffers and the gil. A testcase
>>>>> explicitly
>>>>> for them would be good.
>>>>>
>>>>> Firstprivate etc: i think it'd be nice myself, but it is probably better
>>>>> to
>>>>> take a break from it at this point so that we can think more about that
>>>>> and
>>>>> not do anything rash; perhaps open up a specific thread on them and ask
>>>>> for
>>>>> more general input. Perhaps you want to take a break or task-switch to
>>>>> something else (fused types?) until I can get around to review and merge
>>>>> what you have so far? You'll know best what works for you though. If you
>>>>> decide to implement explicit threadprivate variables because you've got
>>>>> the
>>>>> flow I certainly wom't object myself.
>>>>>
>>>>   Ok, cool, I'll move on :) I already included a test with a prange and
>>>> a numpy buffer with indexing.
>>>
>>> Wow, you're just plowing away at this. Very cool.
>>>
>>> +1 to disallowing nested prange, that seems to get really messy with
>>> little benefit.
>>>
>>> In terms of the CEP, I'm still unconvinced that firstprivate is not
>>> safe to infer, but lets leave the initial values undefined rather than
>>> specifying them to be NaNs (we can do that as an implementation if you
>>> want), which will give us flexibility to change later once we've had a
>>> chance to play around with it.
>>
>> I don't see any technical issues with inferring firstprivate, the question
>> is whether we want to. I suggest not inferring it in order to make this
>> safer: One should be able to just try to change a loop from "range" to
>> "prange", and either a) have things fail very hard, or b) just work
>> correctly and be able to trust the results.
>>
>> Note that when I suggest using NaN, it is as initial values for EACH
>> ITERATION, not per-thread initialization. It is not about "firstprivate" or
>> not, but about disabling thread-private variables entirely in favor of
>> "per-iteration" variables.
>>
>> I believe that by talking about "readonly" and "per-iteration" variables,
>> rather than "thread-shared" and "thread-private" variables, this can be used
>> much more safely and with virtually no knowledge of the details of
>> threading. Again, what's in my mind are scientific programmers with (too)
>> little training.
>>
>> In the end it's a matter of taste and what is most convenient to more users.
>> But I believe the case of needing real thread-private variables that
>> preserves per-thread values across iterations (and thus also can possibly
>> benefit from firstprivate) is seldomly enough used that an explicit
>> declaration is OK, in particular when it buys us so much in safety in the
>> common case.
>>
>> To be very precise,
>>
>> cdef double x, z
>> for i in prange(n):
>>     x = f(x)
>>     z = f(i)
>>     ...
>>
>> goes to
>>
>> cdef double x, z
>> for i in prange(n):
>>     x = z = nan
>>     x = f(x)
>>     z = f(i)
>>     ...
>>
>> and we leave it to the C compiler to (trivially) optimize away "z = nan".
>> And, yes, it is a stopgap solution until we've got control flow analysis so
>> that we can outright disallow such uses of x (without threadprivate
>> declaration, which also gives firstprivate behaviour).
>>
>
> I think the preliminary OpenMP support is ready for review. It
> supports 'with cython.parallel.parallel:' and 'for i in
> cython.parallel.prange(...):'. It works in generators and closures and
> the docs are updated. Support for break/continue/with gil isn't there
> yet.
>
> There are two remaining issue. The first is warnings for potentially
> uninitialized variables for prange(). When you do
>
> for i in prange(start, stop, step): ...
>
> it generates code like
>
> nsteps = (stop - start) / step;
> #pragma omp parallel for lastprivate(i)
> for (temp = 0; temp<  nsteps; temp++) {
>      i = start + temp * step;
>      ...
> }
>
> So here it will complain about 'i' being potentially uninitialized, as
> it might not be assigned to in the loop. However, simply assigning 0
> to 'i' can't work either, as you expect zero iterations not to touch
> it. So for now, we have a bunch of warnings, as I don't see a
> __attribute__ to suppress it selectively.

Isn't this is orthogonal to OpenMP -- even if it said "range", your 
testcase could get such a warning? If so, the fix is simply to 
initialize i in your testcase code.

> The second is NaN-ing private variables, NaN isn't part of C. For gcc,
> the docs ( http://www.delorie.com/gnu/docs/glibc/libc_407.html ) have
> the following to say:
>
> "You can use `#ifdef NAN' to test whether the machine supports NaN.
> (Of course, you must arrange for GNU extensions to be visible, such as
> by defining _GNU_SOURCE, and then you must include `math.h'.)"
>
> So I'm thinking that if NaN is not available (or the compiler is not
> GCC), we can use FLT_MAX, DBL_MAX and LDBL_MAX instead from float.h.
> Would this be the proper way to handle this?

I think it is sufficient. A relatively portable way would be to 
initialize a double variable to 0.0/0.0 at program startup; a problem is 
that that would flag exceptions in the FPU though.

Here's some more compiler-specific stuff I found:

http://www.koders.com/c/fid6EF58B6683BCD810AE371607818952EB039CBC32.aspx

DS