The future of "frozen" types as the number of CPU cores increases

Paul Boddie paul at boddie.org.uk
Sun Feb 21 13:28:46 EST 2010


On 21 Feb, 03:00, "sjdevn... at yahoo.com" <sjdevn... at yahoo.com> wrote:
> On Feb 18, 2:58 pm, John Nagle <na... at animats.com> wrote:
>
> >     Multiple processes are not the answer.  That means loading multiple
> > copies of the same code into different areas of memory.  The cache
> > miss rate goes up accordingly.
>
> A decent OS will use copy-on-write with forked processes, which should
> carry through to the cache for the code.

True, but the principal issue with CPython and copy-on-write is the
reference counting. Consider a large list shared with the child
processes which is to be processed (either in part or in its entirety)
by each of them. As soon as the child processes start referencing each
of the elements, the reference count for each element object will need
to be changed and pages will start to be copied for each child
process. That's why John wants to avoid the reference counting of
shared data and why the Unladen Swallow project has apparently
considered storing reference counts in separate regions of memory.

Such shared data is really "owned" by the parent process, so it makes
little sense for the child processes to manage it with a view to
collecting it later. After all, such data should have no cost to each
of the child processes (except, perhaps, for the size of the address
space referenced by each process, and any resource issues arising from
managing that).

Paul



More information about the Python-list mailing list