[IPython-dev] Musings: syntax for high-level expression of parallel (and other) execution control

[Brian Granger]

Another link that is interesting when you start to think about bytecode
transformations:

http://www.voidspace.org.uk/python/articles/code_blocks.shtml

Cheers,

Brian

On Tue, Sep 8, 2009 at 8:09 PM, Brian Granger <ellisonbg.net at gmail.com>wrote:

>
>
> On Tue, Sep 8, 2009 at 1:34 PM, Fernando Perez <fperez.net at gmail.com>wrote:
>
>> On Tue, Sep 8, 2009 at 10:23 AM, Brian Granger <ellisonbg.net at gmail.com>
>> wrote:
>> > My only complaint is that is a bit unexpected that this actually
>> declares
>> > and *calls* the function!
>>
>> Yes, it is more than a bit surprising at first :)  In fact, I remember
>> that bothered me about @interact when I first saw it, and that was
>> partly why I tried to make things work using 'with'.  But it's now
>> clear to me that we need  a *real* scope for these ideas, and for now,
>> 'def' is the only way to get a scope we have, so that's what we'll be
>> using.
>>
>> For reference, the stuff I was trying to implement from the Ars review
>> is below in the original Objective C code, plus my own 'pythonization'
>> of it, first in serial mode, then using 'with', then with @decos.
>>
>> This is the page in the review where John Siracusa explains the new
>> Block syntax that Apple introduced to C:
>>
>> http://arstechnica.com/apple/reviews/2009/08/mac-os-x-10-6.ars/10
>>
>> I think it's really cool, I do hope it makes its way into the language
>> itself.
>>
>>
> Yes, this is a very nice article.  It is very nice to see Apple bringing
> the "iPod"
> style of thinking (stepping back and really thinking about something) to
> parallelism and languages.
>
> def analyzeDocument():
>>    """A decorator-based version.
>>
>>    This could in principle work just fine today.  All one would need to do
>>    would be to write Python decorators for Apple GCD.  As long as the code
>>    being run released the GIL, it would work fine. """
>>
>>    @dispatch_async(dispatch_get_global_queue(0, 0))
>>    def outer():
>>        stats = myDoc.analyze()
>>        @dispatch_async(dispatch_get_main_queue())
>>        def inner():
>>            myModel.setDict(stats)
>>            myStatsView.setNeedsDisplay(YES)
>>            stats.release()
>>
>
> OK, just a few comments before I loose a week or two thinking about
> this....
>
> * While multiprocessing is interesting, by sticking with the fork model
> of getting variables to child execution contexts, it really misses the
> greater
> possibilities that proper scoping gives you.  With these GCD style
> constructs
> you can do arbitrarily deep nestings of scopes and carefully control what
> variables
> from the parent scope are seen by children scopes.  This is exactly like
> Cilk and
> is a fantastic way of doing things.
>
> * Damn the GIL!  If we didn't have the GIL, we could write a simple version
> of
> dispatch_async that used a simple thread pool we would be off to the races.
>
> * But, we can implement this on top of IPython.kernel and get many of the
> same
> things.  The only real downside is the performance hit of using processes
> rather
> than threads.
>
> Cheers,
>
> Brian
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20090908/e7e13bfb/attachment.html>