Can I get the total memory usage of Python?

[Tim Peters]

[Tim]
>> That's a different question:  malloc and pymalloc requests are
>> untyped -- they have no idea what the memory is used for.  You may
>> like the stats you can get in a Python COUNT_ALLOCS build instead:
>> that keeps track of the current total number of allocations, frees,
>> and highwater mark (max value of #allocations - #frees over time) on
>> a per-type basis (so, e.g., the stats for ints are distinct from the
>> stats for floats).  That's been available "forever", but it's a
>> special build because it's not particularly cheap to keep track of
>> this stuff.

[Cameron Laird]
> No, indeed.  I understand that expense.  Hmmmmmm;
> I suspect the usual result--the hardest part about
> this idea will be to decide what I (we) want.  All
> the possibilities have unsavory aspects.

They do, but also have favorable aspects.  I doubt that thinking about it
will help much before people *try* the various options that already exist,
and see whether they help in practice.  COUNT_ALLOCS in particular is
virtually unknown yet very helpful, and does *not* require a debug build too
(COUNT_ALLOCS is orthogonal to debug-versus-release).  It allocates extra
memory for type objects only, and there are rarely more than a few hundred
of those, so the memory burden is trivial.  The runtime burden is a bit of
extra arithmetic at object creation and destruction times, plus whatever
optimization is lost due to generating extra code at each Py_DECREF site to
increase the per-type "# of frees" count when the refcount hits 0.  In all,
it's not a major expense on any count.

In 2.2.2 and 2.3 it's much more helpful than before, because each
user-defined class counts as "a type", so you can get counts of how many
Frobnicator instances are currently alive, the maximum # ever alive, and the
number that have been destroyed.  Before 2.2, all instances of all
user-defined classes were of the single InstanceType type, so all the
per-class counts got smushed together.

There's more on this in Misc/SpecialBuilds.txt.