Embedding Python, threading and scalability

Andrew Dalke adalke at mindspring.com
Wed Jul 9 00:17:04 CEST 2003

Wenning Qiu:
> I am researching issues related to emdedding Python in C++ for a
> project.

> Has anyone on this list run into the same problem that I have, or does
> anyone know of any plan of totally insulating multiple embedded Python
> interpreters?

Ahh, the Global Interpreter Lock (GIL).

Years ago, Greg Stein had a version of Python 1.4 running with no GIL.

Search for "free threading" to get more hits on this topic.

As I recalled, it slowed down the performance on
machines, so the general take was to keep the GIL.  In addition, see
Tim Peters:
] The prospects for another version of that grow dimmer.  Everyone (incl.
] Greg) has noticed that CPython internals, over time, increase their
] on the thread-safety guarantees of the global interpreter lock.

The only solutions I know of are explicit multi-process solutions:
  - a generic system, like XML-RPC/SOAP/PVM/MPI/CORBA, on
       which you build your own messaging system
  - use systems like Pyro or Twisted, which understand Python objects
       and implement 'transparent' proxying via network communications
  - use POSH, which does the proxying through shared memory (but
       this uses Intel-specific assembly)

                    dalke at dalkescientific.com

More information about the Python-list mailing list