Eric Snow schrieb am 20.06.2015 um 23:42:
tl;dr Let's exploit multiple cores by fixing up subinterpreters, exposing them in Python, and adding a mechanism to safely share objects between them. [...] In some personal correspondence Nick Coghlan, he summarized my preferred approach as "the data storage separation of multiprocessing, with the low message passing overhead of threading".
For Python 3.6:
* expose subinterpreters to Python in a new stdlib module: "subinterpreters" * add a new SubinterpreterExecutor to concurrent.futures * add a queue.Queue-like type that will be used to explicitly share objects between subinterpreters [...] C Extension Modules =================
Subinterpreters already isolate extension modules (and built-in modules, including sys). PEP 384 provides some help too. However, global state in C can easily leak data between subinterpreters, breaking the desired data isolation. This is something that will need to be addressed as part of the effort.
I also had some discussions about these things with Nick before. Not sure if you really meant PEP 384 (you might have) or rather PEP 489: https://www.python.org/dev/peps/pep-0489/ I consider that one more important here, as it will eventually allow Cython modules to support subinterpreters. Unless, as you mentioned, they use global C state, but only in external C code, e.g. wrapped libraries. Cython should be able to handle most of the module internal global state on a per-interpreter basis itself, without too much user code impact. I'm totally +1 for the idea. I hope that I'll find the time (well, and money) to work on PEP 489 in Cython soon, so that I can prove it right for actual real-world code in Python 3.5. We'll then see about subinterpreter support. That's certainly the next step. Stefan