On Thu, Mar 29, 2018, 02:02 Chris Angelico <rosuav@gmail.com> wrote:

On Thu, Mar 29, 2018 at 7:56 PM, Paul Moore <p.f.moore@gmail.com> wrote:
> On 29 March 2018 at 09:49, Chris Angelico <rosuav@gmail.com> wrote:
>> On Thu, Mar 29, 2018 at 7:18 PM, Nathaniel Smith <njs@pobox.com> wrote:
>>> Another example is the multiprocessing module: it's very safe to
>>> assume that the parent and the child are using the same interpreter
>>> :-). There's no fundamental reason you shouldn't be able to send
>>> bytecode between them.
>>
>> You put a smiley on it, but is this actually guaranteed on all
>> platforms? On Unix-like systems, presumably it's using fork() and thus
>> will actually use the exact same binary, but what about on Windows,
>> where a new process has to be spawned? Can you say "spawn me another
>> of this exact binary blob", or do you have to identify it by a file
>> name?
>>
>> It wouldn't be a problem for the nonportable mode to toss out an
>> exception in weird cases like this, but it _would_ be a problem if
>> that causes a segfault or something.
>
> If you're embedding, you need multiprocessing.set_executable()
> (https://docs.python.org/3.6/library/multiprocessing.html#multiprocessing.set_executable),
> so in that case you definitely *won't* have the same binary...

Ah, and that also showed me that forking isn't mandatory on Unix
either. So yeah, there's no assuming that they use the same binary.

Normally it spawns children using `sys.executable`, which I think on Windows in particular is guaranteed to be the same binary that started the main process, because the OS locks the file while it's executing. But yeah, I didn't think about the embedding case, and apparently there's also a little-known set of features for using multiprocessing between arbitrary python processes: https://docs.python.org/3/library/multiprocessing.html#multiprocessing-listeners-clients

I doubt it'll be a problem to pickle though as it'll use some form of
versioning even in NONPORTABLE mode right?

I guess the (merged, but undocumented?) changes in https://bugs.python.org/issue28053 should make it possible to set the pickle version, and yeah, if we did add a NONPORTABLE mode then presumably it would have some kind of header saying which version of python it was created with, so version mismatches could give a sensible error message.

-n