Embedding Python, threading and scalability
adalke at mindspring.com
Wed Jul 9 00:17:04 CEST 2003
> I am researching issues related to emdedding Python in C++ for a
> Has anyone on this list run into the same problem that I have, or does
> anyone know of any plan of totally insulating multiple embedded Python
Ahh, the Global Interpreter Lock (GIL).
Years ago, Greg Stein had a version of Python 1.4 running with no GIL.
Search for "free threading" to get more hits on this topic.
As I recalled, it slowed down the performance on
machines, so the general take was to keep the GIL. In addition, see
] The prospects for another version of that grow dimmer. Everyone (incl.
] Greg) has noticed that CPython internals, over time, increase their
] on the thread-safety guarantees of the global interpreter lock.
The only solutions I know of are explicit multi-process solutions:
- a generic system, like XML-RPC/SOAP/PVM/MPI/CORBA, on
which you build your own messaging system
- use systems like Pyro or Twisted, which understand Python objects
and implement 'transparent' proxying via network communications
- use POSH, which does the proxying through shared memory (but
this uses Intel-specific assembly)
dalke at dalkescientific.com
More information about the Python-list