[Python-ideas] [Python-Dev] PyParallel: alternate async I/O and GIL removal

[Andrew Barnert]

On Nov 17, 2013, at 2:30, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Sun, 17 Nov 2013 16:35:23 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
>> 
>> I think there's some interesting potential in Trent's PyParallel work,
>> but it needs something analogous to Rust's ability to transfer object
>> ownership between threads (thus enabling message passing) to expand
>> beyond the simple worker thread model which is really only interesting
>> on Windows (where processes are expensive - on *nix, processes are
>> generally cheap enough that PyParallel is unlikely to be worth the
>> hassle).
> 
> This is a bit of an oversimplification. The cost of processes is not
> only the cost of spawning them. There is also the CPU cost of
> marshalling data between processes, and the memory cost of having
> duplicate structures and data in your various processes.

But PyParallel doesn't seem to provide _any_ way to pass data between threads. So, the fact that multiprocessing provides only a slow way to pass data between processes can't be considered a weakness. Any program that could be written in PyParallel can't see those costs.