Adding a Par construct to Python?

[Grant Edwards]

On 2009-05-20, Steven D'Aprano <steven at REMOVE.THIS.cybersource.com.au> wrote:

>> Any Python function that isn't calling a library function written in C
>> that releases the GIL won't show any speedup will it?
>
> Not necessarily. Here's another function, that uses a loop instead of 
> sleep.
>
> def g(arg, SIZE=8*10**6):
>     # Default SIZE is chosen so that on my machine, the loop 
>     # takes approximately 0.5 second.
>     for x in xrange(SIZE):
>         pass
>     return 3*arg-2
>
>
>>>> setup = 'from __main__ import pmap, g; data = range(50)'
>>>> min(Timer('map(g, data)', setup).repeat(repeat=5, number=3))
> 65.093590974807739
>>>> min(Timer('pmap(g, data)', setup).repeat(repeat=5, number=3))
> 20.268381118774414

I find that surprising.  Evidently my understanding of the GIL
is wrong.

>> I don't have a multi-core machine to try it on, but what
>> happens when you replace your "slow function" code with
>> something that actually burns CPU using pure-Python code
>> instead of blocking on a timer in the OS?
>
> Two simple work-arounds are:
>
> * use Jython or IronPython; or
>
> * insert time.sleep(0.000001) into your function at various points.

While the latter will allow thread switching, I don't see how
it will increase thread parallelism.

-- 
Grant Edwards                   grante             Yow! WHO sees a BEACH BUNNY
                                  at               sobbing on a SHAG RUG?!
                               visi.com