On 23 June 2015 at 10:03, Chris Angelico <rosuav@gmail.com> wrote:
On Tue, Jun 23, 2015 at 9:59 AM, Nathaniel Smith <njs@pobox.com> wrote:
One possibility would be for subinterpreters to copy modules from the main interpreter -- I guess your average module is mostly dicts, strings, type objects, and functions; strings and functions are already immutable and could be shared without copying, and I guess copying the dicts and type objects into the subinterpreter is much cheaper than hitting the disk etc. to do a real import. (Though certainly not free.)
FWIW, functions aren't immutable, but code objects are.
Anything we come up with for optimised data sharing via channels could be applied to passing a prebuilt sys.modules dictionary through to subinterpreters. The key for me is to start from a well-defined "shared nothing" semantic model, but then look for ways to exploit the fact that we actually *are* running in the same address space to avoid copy objects. The current reference-counts-embedded-in-the-object-structs memory layout also plays havoc with the all-or-nothing page level copy-on-write semantics used by the fork() syscall at the operating system layer, so some of the ideas we've been considering (specifically, those related to moving the reference counter bookkeeping out of the object structs themselves) would potentially help with that as well (but would also have other hard to predict performance consequences). There's a reason Eric announced this as the *start* of a research project, rather than as a finished proposal - while it seems conceptually sound overall, there are a vast number of details to be considered that will no doubt hold a great many devils :) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia