[Python-Dev] Updated PEP 454 (tracemalloc): no more metrics!

[Charles-François Natali]

2013/10/24 Kristján Valur Jónsson <kristjan at ccpgames.com>:
>
> Now, I would personally not truncate the stack, because I can afford the memory,
> but even if I would, for example, to hide a bunch of detail, I would want to throw away
> the _lower_ detals of the stack.  It is unimportant to me to know if memory was
> allocated in
> ...;itertools.py;logging.py;stringutil.py
> but more important to know that it was allocated in
> main.py;databaseengine.py;enginesettings.py;...

Well, maybe to you, but if you look at valgrind for example, it keeps
the top of the stack: and it makes a lot of sense to me, since
otherwise you won't be able to find where the leak occurred.

Anyway, since the stack depth is a tunable parameter, this shouldn't
be an issue in practice: just save the whole stack.


2013/10/24 MRAB <python at mrabarnett.plus.com>:
>
> When I was looking for memory leaks in the regex module I simply wrote all
> of the allocations, reallocations and deallocations to a log file and then
> parsed it afterwards using a Python script. Simple, but effective.

We've all done that ;-)

> 1) really, all that is required in terms of data is the traceback.get_traces() function.  Further, it _need_ not return addresses since they are not required for analysis.  It is sufficient for it to return a list of (traceback, size, count) tuples.

Sure. Since the beginning, I'm also leaning towards a minimal API, and
let third-party tools do the analysis.

It makes a lot of sense, since some people will want just basic
snapshot information, some others will want to compute various
statistics, some others will want to display the result in a GUI...

But OTOT, it would be too bad not to ship the stdlib with a basic tool
to process data, to as to make it usable out-of-the box.

And in this regard, we should probably mimick what's done for CPU
profiling: there are both low-level profiling data gathering
infrastructure (profile and cProfile), but there's also a pstats.Stats
class allowing basic operations/display on this raw data.

That's IMO a reasonable balance.

cf