[Cython] CEP: prange for parallel loops

[Dag Sverre Seljebotn]

On 04/04/2011 03:04 PM, Stefan Behnel wrote:
> Dag Sverre Seljebotn, 04.04.2011 13:53:
>> On 04/04/2011 01:23 PM, Stefan Behnel wrote:
>>> Dag Sverre Seljebotn, 04.04.2011 12:17:
>>>> CEP up at http://wiki.cython.org/enhancements/prange
>>>
>>> """
>>> Variable handling
>>>
>>> Rather than explicit declaration of shared/private variables we rely on
>>> conventions:
>>>
>>> * Thread-shared: Variables that are only read and not written in the 
>>> loop
>>> body are shared across threads. Variables that are only used in the 
>>> else
>>> block are considered shared as well.
>>>
>>> * Thread-private: Variables that are assigned to in the loop body are
>>> thread-private. Obviously, the iteration counter is thread-private 
>>> as well.
>>>
>>> * Reduction: Variables that only used on the LHS of an inplace 
>>> operator,
>>> such as s above, are marked as targets for reduction. If the 
>>> variable is
>>> also used in other ways (LHS of assignment or in an expression) it does
>>> instead turn into a thread-private variable. Note: This means that if
>>> one, e.g., inserts printf(... s) above, s is turned into a thread-local
>>> variable. OTOH, there is simply no way to correctly emulate the effect
>>> printf(... s) would have in a sequential loop, so such code must be
>>> discouraged anyway.
>>> """
>>>
>>> What about simply (ab-)using Python semantics and creating a new inner
>>> scope for the prange loop body? That would basically make the loop 
>>> behave
>>> like a closure function, but with the looping header at the 'right' 
>>> place
>>> rather than after the closure.
>>
>> I'm not quite sure what the concrete changes to the CEP this would 
>> lead to
>> (assuming you mean this as a proposal for alternative semantics, and 
>> not an
>> implementation detail).
>
> What I would like to avoid is having to tell users "and now for 
> something completely different". It looks like a loop, but then 
> there's a whole page of new semantics for it. And this also cannot be 
> used in plain Python code due to the differing scoping behaviour.

Well, at least it's better than the 300 pages of semantics for OpenMP :-)

>
>
>> How would we treat reduction variables? They need to be supported, and
>> there's nothing in Python semantics to support reduction variables, they
>> are a rather special case everywhere. I suppose keeping the reduction
>> clause above, or use the "nonlocal" keyword in the loop body...
>
> That's what I thought, yes. It looks unexpected, sure. That's the 
> clear advantage of using inner functions, which do not add anything 
> new at all. But if we want to add something that looks more like a 
> loop, we should at least make it behave like something that's easy to 
> explain.
>
> Sorry for not taking the opportunity to articulate my scepticism in 
> the workshop discussion.


I like the idea of considering cdef/nonlocal in the prange blocks. But, 
yes, I do feel that opposing a parallel loop construct in general is 
rather late, or at least could have been done at a more convenient time...

All I know and care about is that a decorator-and-closure solution will 
be a lot more obscure among non-CS people who have no clue what a 
closure or decorator is, and those are exactly the people who need this 
kind of simple 80%-solution.  You and me don't really need any support 
from Cython at all to write multithreaded apps (leaving aesthetics and 
number of keystrokes to the side).

It'd be good to hear Robert's and Mark's opinions before going further, 
let's economise this thread a bit.

Dag Sverre