Exploiting Dual Core's with Py_NewInterpreter's separated GIL ?
filipwasilewski at gmail.com
Fri Nov 3 12:38:43 CET 2006
> I'd like to use multiple CPU cores for selected time consuming Python computations (incl. numpy/scipy) in a frictionless manner.
> Interprocess communication is tedious and out of question, so I thought about simply using a more Python interpreter instances (Py_NewInterpreter) with extra GIL in the same process.
> I expect to be able to directly push around Python Object-Trees between the 2 (or more) interpreters by doing some careful locking.
I don't want to discourage you but what about reference counting/memory
management for shared objects? Doesn't seem fun for me.
Take a look at IPython1 and it's parallel computing capabilities [1,
2]. It is designed to run on multiple systems or a single system with
multiple CPU/multi-core. It's worker interpreters (engines) are loosely
coupled and can utilize several MPI modules, so there is no low-level
messing with GIL. Although it is work in progress it already looks
More information about the Python-list