On Sun, 21 Jun 2015 13:41:30 +0200 Sturla Molden <sturla.molden@gmail.com> wrote:
From the perspective of software design, it would be good it the CPython interpreter provided an environment instead of using global objects. It would mean that all functions in the C API would need to take the environment pointer as their first variable, which will be a major rewrite. It would also allow the "one interpreter per thread" design similar to tcl and .NET application domains.
From the point of view of API compatibility, it's unfortunately a no-no.
The reason IPC in multiprocessing is slow is due to calling pickle, it is not the IPC in itself.
No need to be pedantic :-) The "C" means communication, and pickling objects is part of the communication between Python processes.
All in all, I think we are better off finding a better way to share Python objects between processes.
Sure. This is however a complex and experimental topic (how to share a graph of garbage-collected objects between independant processes), with no guarantees of showing any results at the end.
P.S. Another thing to note is that with sub-interpreters, you can forget about using ctypes or anything else that uses the simplified GIL API (e.g. certain Cython generated extensions).
Indeed, the PyGILState API is still not subinterpreter-compatible. There's a proposal on the tracker, IIRC, but the interested parties never made any progress on it. Regards Antoine.