[Python-ideas] Parallel processing with Python
sturla at molden.no
Thu Feb 19 11:53:30 CET 2009
On 2/19/2009 3:34 AM, Adam Olsen wrote:
> * Erlang's modules/classes/functions are not first-class objects, so
> it doesn't need a copy of them. Python does, so each interpreter
> would have a memory footprint about the same as a true process.
Yes, each interpreter will have the memory footprint of an interpreter.
So the memory use would be about the same as with multiprocessing.
> * Any communication requires a serialize/copy/deserialize sequence.
No it does not, and this why embedded interpreters are better than
multiple processes (cf. multiprocessing). Since the interpreters share
virtual memory, many objects can be shared without any serialization.
That is, C pointers will be valid in both interpreters, so it should in
many cases be possible to pass a PyObject* from one interpreter to
another. This kind of communication would be easiest to achieve with
Another advantage is that there will be just one process to kill,
whenever that is required.
More information about the Python-ideas