Devin Jeanpierre <jeanpierreda@gmail.com> wrote:
The comparison is unfair -- if you can share between subinterpreters using memcpy, then you can share between processes using just a socket write, and multiprocessing becomes nearly just as fast.
That is the main issue here. Writing to a pipe or a Unix socket is implemented with a memcpy in the kernel. So there is just a tiny constant overhead compared to using memcpy within a process. And with shared memory as IPC even this tiny overhead can be removed. The main overhead in communicating Python objects in multiprocessing is the serialization with pickle. So there is basically nothing to gain unless this part can be omitted. There is an errorneous belief among Windows programmers tht "IPC is slow". But that is because they are using out-proc DCOM server, CORBA, XMLRPC or something equally atrocious. A plain named pipe transaction is not in any way slow on Windows. Sturla