Parallel Python

robert no-spam at no-spam-no-spam.invalid
Mon Jan 8 17:23:52 CET 2007

Duncan Booth wrote:
> Laszlo Nagy <gandalf at> wrote:
> The 'parallel python' site seems very sparse on the details of how it is 
> implemented but it looks like all it is doing is spawning some subprocesses 
> and using some simple ipc to pass details of the calls and results. I can't 
> tell from reading it what it is supposed to add over any of the other 
> systems which do the same.
> Combined with the closed source 'no redistribution' license I can't really 
> see anyone using it.

Thats true. IPC through sockets or (somewhat faster) shared memory -  cPickle at least - is usually the maximum of such approaches.

For tasks really requiring threading one can consider IronPython.
Most advanced technique I've see for CPython ist posh : 

I'd say Py3K should just do the locking job for dicts / collections, obmalloc and refcount (or drop the refcount mechanism) and do the other minor things in order to enable free threading. Or at least enable careful sharing of Py-Objects between multiple separated Interpreter instances of one process.
.NET and Java have shown that the speed costs for this technique are no so extreme. I guess less than 10%. 
And Python is a VHLL with less focus on speed anyway.
Also see discussions in .


More information about the Python-list mailing list