2013/8/31 Gregory P. Smith <greg@krypto.org>:
Think of the possibilities, you could even setup a test runner to enable/disable before and after each test, test suite or test module to gather narrow statistics as to what code actually _caused_ the allocations rather than the ultimate individual file/line doing it.
It may be used with "-m test -R" to detect memory leaks, you need to repeat a test.
Taking that further: file and line information is great, but what if you extend the concept: could you allow for C API or even Python hooks to gather additional information at the time of each allocation or free? for example: Gathering the actual C and Python stack traces for correlation to figure out what call patterns lead allocations is powerful.
I modified the PEP 454 and the implementation. By default, tracemalloc only stores 1 "frame" instance (filename and line number). But you can now use tracemalloc.set_number_frame(10) to store 10 frames per memory allocation. Don't store too much frames, remember that tracemalloc traces all Python memory allocations! So when Python allocates 1 byte, tracemalloc may need to allocate 1000 bytes to store the trace... tracemalloc cannot get the C traceback. It was discussed in the design of the PEP 445 (Add new APIs to customize Python memory allocators) how to pass the C filename and line number. It was decided to not add them to have a simpler API. In general, a C function is binded to a Python function. Knowing the Python function should be enough. Victor