duncan.booth at invalid.invalid
Mon Jan 8 11:33:54 CET 2007
Laszlo Nagy <gandalf at designaproduct.biz> wrote:
> parallelpython at gmail.com wrote:
>> Has anybody tried to run parallel python applications?
>> It appears that if your application is computation-bound using 'thread'
>> or 'threading' modules will not get you any speedup. That is because
>> python interpreter uses GIL(Global Interpreter Lock) for internal
>> bookkeeping. The later allows only one python byte-code instruction to
>> be executed at a time even if you have a multiprocessor computer.
>> To overcome this limitation, I've created ppsmp module:
>> It provides an easy way to run parallel python applications on smp
>> I would appreciate any comments/suggestions regarding it.
> I always thought that if you use multiple processes (e.g. os.fork) then
> Python can take advantage of multiple processors. I think the GIL locks
> one processor only. The problem is that one interpreted can be run on
> one processor only. Am I not right? Is your ppm module runs the same
> interpreter on multiple processors? That would be very interesting, and
> something new.
The GIL locks all processors, but just for one process. So, yes, if you
spawn off multiple processes then Python will take advantage of this. For
example we run Zope on a couple of dual processor dual core systems, so we
use squid and pound to ensure that the requests are spread across 4
instances of Zope on each machine. That way we do get a fairly even cpu
For some applications it is much harder to split the tasks across separate
processes rather than just separate threads, but there is a benefit once
you've done it since you can then distribute the processing across cpus on
The 'parallel python' site seems very sparse on the details of how it is
implemented but it looks like all it is doing is spawning some subprocesses
and using some simple ipc to pass details of the calls and results. I can't
tell from reading it what it is supposed to add over any of the other
systems which do the same.
Combined with the closed source 'no redistribution' license I can't really
see anyone using it.
More information about the Python-list