On 13 September 2017 at 14:10, Nathaniel Smith <njs@pobox.com> wrote:
Subinterpreters are basically an attempt to reimplement the OS's process isolation in user-space, right?
Not really, they're more an attempt to make something resembling Rust's memory model available to Python programs - having the default behaviour be "memory is not shared", but having the choice to share when you want to be entirely an application level decision, without getting into the kind of complexity needed to deliberately break operating system level process isolation. The difference is that where Rust was able to do that on a per-thread basis and rely on their borrow checker for enforcement of memory ownership, for PEP 554, we're proposing to do it on a per-interpreter basis, and rely on runtime object space partitioning (where Python objects and the memory allocators are *not* shared between interpreters) to keep things separated from each other. That's why memoryview is such a key part of making the proposal interesting: it's what lets us relatively easily poke holes in the object level partitioning between interpreters and provide zero-copy messaging passing without having to share any regular reference counts between interpreters (which in turn is what makes it plausible that we may eventually be able to switch to a true GIL-per-interpreter model, with only a few cross-interpreter locks for operations like accessing the list of interpreters itself). Right now, the closest equivalent to this programming model that Python offers is to combine threads with queue.Queue, and it requires a lot of programming discipline to ensure that you don't access an object again once you've submitted to a queue. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia