[Python-ideas] Python 3000 TIOBE -3%

[Massimo Di Pierro]

On Feb 10, 2012, at 9:43 AM, Arnaud Delobelle wrote:

> On 10 February 2012 14:52, Massimo Di Pierro <massimo.dipierro at gmail.com> wrote:
>> Forking is a solution only for simple toy cases and in trivially parallel
>> cases. People use processes to parallelize web serves and task queues where
>> the tasks do not need to talk to each other (except with the parent/master
>> process). If you have 100 cores even with a small 50MB program, in order to
>> parallelize it you go from 50MB to 5GB. Memory and memory access become a
>> major bottle neck.
> 
> I don't know much about forking, but I'm pretty sure that forking a
> process doesn't mean you double the amount of physical memory used.
> With copy-on-write, a lot of physical memory can be shared.

Anyway, copy-on-write does not solve the problem. The OS tries to save memory but not duplicating physical memory space and by assigning the different address spaces of the various forked processes to the same physical memory. But as soon as one process writes into the segment, the entire segment is copied. It has to be, the processes must have different address spaces. That is what fork does.

Anyway, there are many applications that are parallelized well with processes (at least for a small number of cores/cpus).


> 
> -- 
> Arnaud