[Cython] prange CEP updated

[mark florisson]

On 4 May 2011 12:45, Dag Sverre Seljebotn <d.s.seljebotn at astro.uio.no> wrote:
> On 05/04/2011 12:00 PM, mark florisson wrote:
>>
>> On 21 April 2011 20:13, Dag Sverre Seljebotn<d.s.seljebotn at astro.uio.no>
>>  wrote:
>>>
>>> On 04/21/2011 10:37 AM, Robert Bradshaw wrote:
>>>>
>>>> On Mon, Apr 18, 2011 at 7:51 AM, mark florisson
>>>> <markflorisson88 at gmail.com>    wrote:
>>>>>
>>>>> On 18 April 2011 16:41, Dag Sverre
>>>>> Seljebotn<d.s.seljebotn at astro.uio.no>
>>>>>  wrote:
>>>>>>
>>>>>> Excellent! Sounds great! (as I won't have my laptop for some days I
>>>>>> can't
>>>>>> have a look yet but I will later)
>>>>>>
>>>>>> You're right about (the current) buffers and the gil. A testcase
>>>>>> explicitly
>>>>>> for them would be good.
>>>>>>
>>>>>> Firstprivate etc: i think it'd be nice myself, but it is probably
>>>>>> better
>>>>>> to
>>>>>> take a break from it at this point so that we can think more about
>>>>>> that
>>>>>> and
>>>>>> not do anything rash; perhaps open up a specific thread on them and
>>>>>> ask
>>>>>> for
>>>>>> more general input. Perhaps you want to take a break or task-switch to
>>>>>> something else (fused types?) until I can get around to review and
>>>>>> merge
>>>>>> what you have so far? You'll know best what works for you though. If
>>>>>> you
>>>>>> decide to implement explicit threadprivate variables because you've
>>>>>> got
>>>>>> the
>>>>>> flow I certainly wom't object myself.
>>>>>>
>>>>>  Ok, cool, I'll move on :) I already included a test with a prange and
>>>>> a numpy buffer with indexing.
>>>>
>>>> Wow, you're just plowing away at this. Very cool.
>>>>
>>>> +1 to disallowing nested prange, that seems to get really messy with
>>>> little benefit.
>>>>
>>>> In terms of the CEP, I'm still unconvinced that firstprivate is not
>>>> safe to infer, but lets leave the initial values undefined rather than
>>>> specifying them to be NaNs (we can do that as an implementation if you
>>>> want), which will give us flexibility to change later once we've had a
>>>> chance to play around with it.
>>>
>>> I don't see any technical issues with inferring firstprivate, the
>>> question
>>> is whether we want to. I suggest not inferring it in order to make this
>>> safer: One should be able to just try to change a loop from "range" to
>>> "prange", and either a) have things fail very hard, or b) just work
>>> correctly and be able to trust the results.
>>>
>>> Note that when I suggest using NaN, it is as initial values for EACH
>>> ITERATION, not per-thread initialization. It is not about "firstprivate"
>>> or
>>> not, but about disabling thread-private variables entirely in favor of
>>> "per-iteration" variables.
>>>
>>> I believe that by talking about "readonly" and "per-iteration" variables,
>>> rather than "thread-shared" and "thread-private" variables, this can be
>>> used
>>> much more safely and with virtually no knowledge of the details of
>>> threading. Again, what's in my mind are scientific programmers with (too)
>>> little training.
>>>
>>> In the end it's a matter of taste and what is most convenient to more
>>> users.
>>> But I believe the case of needing real thread-private variables that
>>> preserves per-thread values across iterations (and thus also can possibly
>>> benefit from firstprivate) is seldomly enough used that an explicit
>>> declaration is OK, in particular when it buys us so much in safety in the
>>> common case.
>>>
>>> To be very precise,
>>>
>>> cdef double x, z
>>> for i in prange(n):
>>>    x = f(x)
>>>    z = f(i)
>>>    ...
>>>
>>> goes to
>>>
>>> cdef double x, z
>>> for i in prange(n):
>>>    x = z = nan
>>>    x = f(x)
>>>    z = f(i)
>>>    ...
>>>
>>> and we leave it to the C compiler to (trivially) optimize away "z = nan".
>>> And, yes, it is a stopgap solution until we've got control flow analysis
>>> so
>>> that we can outright disallow such uses of x (without threadprivate
>>> declaration, which also gives firstprivate behaviour).
>>>
>>
>> I think the preliminary OpenMP support is ready for review. It
>> supports 'with cython.parallel.parallel:' and 'for i in
>> cython.parallel.prange(...):'. It works in generators and closures and
>> the docs are updated. Support for break/continue/with gil isn't there
>> yet.
>>
>> There are two remaining issue. The first is warnings for potentially
>> uninitialized variables for prange(). When you do
>>
>> for i in prange(start, stop, step): ...
>>
>> it generates code like
>>
>> nsteps = (stop - start) / step;
>> #pragma omp parallel for lastprivate(i)
>> for (temp = 0; temp<  nsteps; temp++) {
>>     i = start + temp * step;
>>     ...
>> }
>>
>> So here it will complain about 'i' being potentially uninitialized, as
>> it might not be assigned to in the loop. However, simply assigning 0
>> to 'i' can't work either, as you expect zero iterations not to touch
>> it. So for now, we have a bunch of warnings, as I don't see a
>> __attribute__ to suppress it selectively.
>
> Isn't this is orthogonal to OpenMP -- even if it said "range", your testcase
> could get such a warning? If so, the fix is simply to initialize i in your
> testcase code.

No, the problem is that 'i' needs to be lastprivate, and 'i' is
assigned to in the loop body. It's irrelevant whether 'i' is assigned
to before the loop. I think this is the case because the spec says
that lastprivate variables will get the value of the private variable
of the last sequential iteration, but it cannot at compile time know
whether there might be zero iterations, which I believe the spec
doesn't have anything to say about. So basically we could guard
against it by checking if nsteps > 0, but the compiler doesn't detect
this, so it will still issue a warning even if 'i' is initialized (the
warning is at the place of the lastprivate declaration).

>> The second is NaN-ing private variables, NaN isn't part of C. For gcc,
>> the docs ( http://www.delorie.com/gnu/docs/glibc/libc_407.html ) have
>> the following to say:
>>
>> "You can use `#ifdef NAN' to test whether the machine supports NaN.
>> (Of course, you must arrange for GNU extensions to be visible, such as
>> by defining _GNU_SOURCE, and then you must include `math.h'.)"
>>
>> So I'm thinking that if NaN is not available (or the compiler is not
>> GCC), we can use FLT_MAX, DBL_MAX and LDBL_MAX instead from float.h.
>> Would this be the proper way to handle this?
>
> I think it is sufficient. A relatively portable way would be to initialize a
> double variable to 0.0/0.0 at program startup; a problem is that that would
> flag exceptions in the FPU though.
>
> Here's some more compiler-specific stuff I found:
>
> http://www.koders.com/c/fid6EF58B6683BCD810AE371607818952EB039CBC32.aspx

Thanks, I'll take a look!

> DS
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel
>