[Python-Dev] Updated PEP 454 (tracemalloc): no more metrics!

[Victor Stinner]

2013/10/26 Kristján Valur Jónsson <kristjan at ccpgames.com>:
> In that case, how about adding a client/server feature?
>
> If you standardize the format, a minimal tracing client could write a log,
> or send it to a socket, in a way that can be turned into a snapshot by a
> corresponsing utility reading from a file or listenting to a socket.
>
> Just a though.  Could be a future addition…

tracemalloc maintains a dictionary of all allocated memory blocks,
which is slow and eats a lot of memory. You don't need tracemalloc to
log calls to malloc/realloc/free. You can write your own hook using
the PEP 445 (malloc API). A code just writing to stderr should not be
longer than 100 linues (tracemalloc is closer to 2500 lines).

As I wrote, I began to log all calls into a file, but it was too slow.
The memory leak occurred after between 2 hours and 1 day. To know the
current state (ex: total memory currently allocated), you have to
parse the whole log file, which can be really huge. Writing the log
file, transfering the log file and analyzing the log file are too slow
were too slow in my use cases.

Victor