[Python-Dev] Add a new tracemalloc module to trace memory allocations

[Victor Stinner]

2013/8/29 Victor Stinner <victor.stinner at gmail.com>:
> My proposed implementation for Python 3.4 is different:
>
> * no enable() / disable() function: tracemalloc can only be enabled
> before startup by setting PYTHONTRACEMALLOC=1 environment variable
>
> * traces (size of the memory block, Python filename, Python line
> number) are stored directly in the memory block, not in a separated
> hash table
>
> I chose PYTHONTRACEMALLOC env var instead of enable()/disable()
> functions to be able to really trace *all* memory allocated by Python,
> especially memory allocated at startup, during Python initialization.

I'm not sure that having to set an environment variable is the most
convinient option, especially on Windows.

Storing traces directly into memory blocks should use less memory, but
it requires to start tracemalloc before the first memory allocation.
It is possible to add again enable() and disable() methods to
dynamically install/uninstall the hook on memory allocators. I solved
this issue in the current implementation by using a second hash table
(pointer => trace).

We can keep the environment variable as PYTHONFAULTHANDLER which
enables faulthandler at startup. faulthandler has also a command line
option: -X faulthandler. We may add -X tracemalloc.

Victor