![](https://secure.gravatar.com/avatar/870d613430249e453343efc9667ef636.jpg?s=120&d=mm&r=g)
On 2020-06-16 20:28, Mark Shannon wrote:
On 16/06/2020 1:24 pm, Nick Coghlan wrote:
Multiprocessing serialisation overheads are abysmal. With enough OS support you can attempt to mitigate that via shared memory mechanisms (which Davin added to the standard library), but it's impossible to get the overhead of doing that as low as actually using the address space of one OS process.
What does "multiprocessing serialisation" even mean? I assume you mean the overhead of serializing objects for communication between processes.
The cost of serializing an object has absolutely nothing to do with which process the interpreter is running in.
Separate interpreters within a single process will still need to serialize objects for communication.
The overhead of passing data through shared memory is the same for threads and processes. It's just memory.
Can we please stick to facts and not throw around terms like "abysmal" with no data whatsoever to back it up.
I'd like to get back to the facts. Let me quote the original mail from this thread: On 2020-06-05 16:32, Mark Shannon wrote:
While I'm in favour of PEP 554, or some similar model for parallelism in Python, I am opposed to the changes we are currently making to support it.
Which changes? There are several efforts in this general space. Personally, I also don't agree with them all. And I think the reason I wasn't able to formulate too many replies to you is that we don't have a common understanding of what is being discussed, and of the modivations behind the changes. You seem to try convince everyone that multiple processes are better (at isolation, and at performance) than multiple interpreters in one process. And I see the point: if you can live with the restriction of multiple processes, they probably are a better choice! But I don't think PEPs 554, 489, 573, etc. are about choosing between multiprocessing and multiple interpreters; they're about making multiple interpreters better than they currently are.