[Python-Dev] Investigating Python memory footprint of one real Web application

[Christian Heimes]

On 2017-01-20 13:15, INADA Naoki wrote:
>>
>> "this script counts static memory usage. It doesn’t care about dynamic
>> memory usage of processing real request"
>>
>> You may be trying to optimize something which is only a very small
>> fraction of your actual memory footprint.  That said, the marshal
>> module could certainly try to intern some tuples and other immutable
>> structures.
>>
> 
> Yes.  I hadn't think static memory footprint is so important.
> 
> But Instagram tried to increase CoW efficiency of prefork application,
> and got some success about memory usage and CPU throughput.
> I surprised about it because prefork only shares static memory footprint.
> 
> Maybe, sharing some tuples which code object has may increase cache efficiency.
> I'll try run pyperformance with the marshal patch.

IIRC Thomas Wouters (?) has been working on a patch to move the ref
counter out of the PyObject struct and into a dedicated memory area. He
proposed the idea to improve cache affinity, reduce cache evictions and
to make CoW more efficient. Especially modern ccNUMA machines with
multiple processors could benefit from the improvement, but also single
processor/multi core machines.

Christian