
It's worthwhile to consider fork as an alternative. IMO we'd get a lot out of making forking safer, easier, and more efficient. (e.g. respectively: adding an atfork registration mechanism; separating out the bits of multiprocessing that use pickle from those that don't; moving the refcount to a separate page, or allowing it to be frozen prior to a fork.) It sounds to me like this approach would use more memory than either regular threaded code or forking, so its main advantages are being cross-platform and less bug-prone. Is that right? Note: I don't count the IPC cost of forking, because at least on linux, any way to efficiently share objects between independent interpreters in separate threads can also be ported to independent interpreters in forked subprocesses, and *should* be. See also: multiprocessing.Value/Array. This is probably a good opportunity for that unification you mentioned. :) On Sat, Jun 20, 2015 at 3:04 PM, Yury Selivanov <yselivanov.ml@gmail.com> wrote:
On 2015-06-20 5:42 PM, Eric Snow wrote:
* only allow immutable objects to be shared between subinterpreters
Even if this is the only thing we have -- an efficient way for sharing immutable objects (such as bytes, strings, ints, and, stretching the definition of immutable, FDs) that will allow us to do a lot.
+1, this has a lot of utility, and can be extended naturally to other types and circumstances. -- Devin