Object counts

[Tim Peters]

[Chuck Esterbrook]
> Is there a way to get Python to report the number of instances of all
> classes and types?
> ...

For types, yes, but it's not documented, and if anyone had time to explain
it they'd probably write docs instead <wink>.  You need to recompile Python
with COUNT_ALLOCS #define'd.  Then see the functions get_counts() and
dump_counts() in Objects/object.c, which are only defined when COUNT_ALLOCS
is.  The sys module also grows a new corresponding Python-level function
getcounts() then, and dump_counts() is automatically called when Python
shuts down then.  COUNT_ALLOCS is rarely used (I've never used it, and,
indeed, the only reason I know it exists is that several years ago somebody
submitted a bug report against it that Guido asked me to fix).

A much bigger gun can be loaded by recompiling Python with Py_TRACE_REFS
defined.  Then a list of almost all live objects at any given time can be
obtained by calling the otherwise non-existent sys.getobjects() function.
It's "almost all" because it only includes objects allocated from the heap;
some builtin type objects are statically allocated instead, so don't show up
in the list.

Note:  Py_TRACE_REFS is automatically enabled in a debug-mode build
(Py_DEBUG implies Py_TRACE_REFS).  I don't think many people know that.  So
in a debug mode build, sys.getobjects exists, and you can build just about
any object-use analyzer on top of that in pure Python.

> e.g., a dictionary that would map the names of types/classes to their
> count,

A fine example <wink>.  It's trivial to build such a dict yourself, by
marching over the list returned by sys.getobjects().

Do note that "almost all objects" can be a lot of objects; here on Windows
running a debug-build Python:

C:\Code\python\PCbuild>python_d
Adding parser accelerators ...
Done.
Python 2.3a0 (#29, Feb 13 2002, 18:39:45) [MSC 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
[7092 refs]
>>> len(sys.getobjects(0))
2459
[7094 refs]
>>>

That is, there are almost 2500 objects alive "immediately" upon startup.
The "0" argument there can be nonzero if you want to clamp the maximum size
of the result list.  An optional second type argument can be specified to
return live objects of only that type:

>>> x = sys.getobjects(0, int) # using 'int' this way requires 2.2
[7155 refs]
>>> len(x)
56
[7155 refs]
>>> x[:10]
[2459, 17, 19, 20, 18, 14, 56, 448, 128, 256]
[7170 refs]
>>> sys.getobjects(10, int)
[2459, 17, 19, 20, 18, 14, 56, 448, 128, 256]
[7170 refs]
>>>

Note that "2459" is still sitting around from the first call I showed in
this session, thanks to the magical _ variable in interactive mode, then
thanks to capturing it in the list x.  After several years, the cause of
such mysteries becomes obvious at first sight <wink>.  Let's get rid of
2459:

>>> del x
[7113 refs]
>>> sys.getobjects(10, int)
[2459, 17, 19, 20, 18, 14, 56, 448, 128, 256]
>>>

It's still there!  Can you guess why?  It's because _ held the result list
from the *preceding* time sys.getobjects(10, int) got called.  Now we'll
nuke it for real:

>>> 3 # get 2459 out of _
3
[7103 refs]
>>> sys.getobjects(10, int)
[17, 19, 20, 18, 14, 56, 448, 128, 256, 512]
[7113 refs]
>>>

Bingo.  You could fritter away many useless hours trying to figure out why
448 is in the list <wink>.