[Python-Dev] CPython optimization: storing reference counters outside of objects

[geremy condra]

On Tue, May 24, 2011 at 8:44 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 5/24/2011 8:25 AM, Sturla Molden wrote:
>
>> Artur Siekielski is not talking about cache locality, but copy-on-write
>> fork on Linux et al.
>>
>> When reference counts are updated after forking, memory pages marked
>> copy-on-write are copied if they store reference counts. And then he
>> quickly runs out of memory. He wants to put reference counts and
>> PyObjects in different pages, so only the pages with reference counts
>> get copied.
>>
>> I don't think he cares about cache locality at all, but the rest of us
>> do :-)
>
> It seems clear that separating reference counts from objects satisfies a
> specialized need and should be done in a spedial, patched version of CPython
> rather than the general distribution.

I'm not sure I agree, especially given that the classical answer to
GIL woes has been to tell people to fork() themselves. There has to be
a lot of code out there that would benefit from this.

Geremy Condra