[Cython] cython.parallel tasks, single, master, critical, barriers

[mark florisson]

On 20 October 2011 10:35, Dag Sverre Seljebotn
<d.s.seljebotn at astro.uio.no> wrote:
> On 10/20/2011 11:13 AM, mark florisson wrote:
>>
>> On 20 October 2011 09:42, Dag Sverre Seljebotn
>> <d.s.seljebotn at astro.uio.no>  wrote:
>>>
>>> Meta: I've been meaning to respond to this thread, but can't find the
>>> time.
>>> What's the time-frame for implementing this? If it's hypothetical at the
>>> moment and just is a question of getting things spec-ed, one could
>>> perhaps
>>> look at discussing it at the next Cython workshop, or perhaps a Skype
>>> call
>>> with the three of us as some point...
>>
>> For me this is just about getting this spec-ed, so that when someone
>> finds the time, we don't need to discuss it for weeks first. And the
>> implementor won't necessarily have to support everything at once, e.g.
>> just critical sections or barriers alone would be nice.
>>
>> Is there any plan for a new workshop then? Because if it's in two
>> years I think we could be more time-efficient :)
>
> At least in William's grant there's plans for 2-3 Cython workshops, so
> hopefully there's funding for one next year if we want to. We should ask him
> before planning anything though.
>
>>> Regarding the tasks: One of my biggest problems with Python is the lack
>>> of
>>> an elegant syntax for anonymous functions. But since Python has that
>>> problem, I feel it is not necesarrily something we should fix (by using
>>> the
>>> with statements to create tasks). Sometimes Pythonic-ness is more
>>> important
>>> than elegance (for Cython).
>>
>> I agree it's not something we should fix, I just think tasks are most
>> useful in inline blocks and not in separate functions or closures.
>> Although it could certainly work, I think it restricts more, leads to
>> more verbose code and possibly questionable semantics, and on top of
>> that it would be a pain to implement (although that should not be used
>> as a persuasive argument). I'm not saying there is no elegant way
>> other than with blocks, I'm just saying that I think closures are not
>> the right thing for it.
>>
>>> In general I'm happy as long as there's a chance of getting things to
>>> work
>>> in pure Python mode as well (with serial execution). So if, e.g., with
>>> statements creating tasks have the same effect when running the same code
>>> (serially) in pure Python, I'm less opposed (didn't look at it in
>>> detail).
>>
>> Yes, it would have the same effect. The thing with tasks (and OpenMP
>> constructs in general) is that usually if your compiler ignores all
>> your pragmas, your code just runs serially in the same way. The same
>> would be true for the tasks in with blocks.
>
> Short note: I like the vision of Konrad Hinsen:
>
> http://www.euroscipy.org/talk/2011
>
> The core idea is that the "task-ness" of a block of code is orthogonal to
> the place you actually write it. That is, a block of code may often either
> be fit for execution as a task, or not, depending on how heavy it is (=
> values of arguments it takes in, not its contents).
>
> He introduces the "async" expression to drive this point through.
>
> I think "with task" is fine if used in this way, if you simply call a
> function (which itself doesn't know whether it is a task or not). But once
> you start to implement an entire function within the with-statement there's
> a code-smell.

Definitely, do you'd just call the function from the task.

> Anyway, it's growing on me. But I think his "async" expression is more
> Pythonic in the way that it forces you away from making your code smell.
>
> We could simply have
>
> async(func)(arg, arg2, somekwarg=4)
>

That looks good. The question is, does this constitute an expression
or a statement? If it's an expression, then you expect a meaningful
return value, which means you're going to have to wait for the task to
complete. That would be fine if you submit multiple tasks in one
expression, from the slides:

    max(async expr1, async expr2)

or even

    [async expr for ... in ...]

I must say, it does look really elegant and it doesn't leave the user
to question when the task is executed (and if you need a taskwait
directive to wait for your variables to become defined). What I don't
see is how to do the producer consumer trick, unless you regard using
the result of async as a taskwait, and not using it as not having a
taskwait, e.g.

async func(...) # generate a task and don't wait for it
result = async func(...) # generate a task and wait for it.

The latter is not useful unless you have multiple expressions in one
statement, so we should also allow result1, result2 = async
func(data=a), async func(data=b).

I think you would need special support for the expression form to work
in multiple places, e.g. as a start you could allow it as function
arguments, tuple expressions and possibly a nogil form of list
comprehensions. The statement form is a lot more simple, and as a
start synchronization must simply be done through barriers. If you
want to change additional data through mechanisms other than immediate
result collection you have to pass in pointers to the data.

I like the syntax of async(func)(arg, arg2, somekwarg=4), as it would
work in pure mode, and you can still have something that looks like a
normal function call, but makes it clear it has to be a function call.
Then at a later point you could decide to support 'with async():' :).

> (He also says "functional-style programming is better for parallization than
> threads+locks", which I can kind of agree with but nobody tried to make an
> efficient immutable array implementation suitable for numerical computation
> yet to my knowledge... that's an interesting MSc or PhD-topic, but I already
> have one :-) )
>
> (Look at me, going along discussing when I really shouldn't -- see you
> later.)
>
> Dag Sverre
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel
>