Where'd my memory go? (was Re: [Python-Dev] Dictionary sparseness)

[Skip Montanaro]

    Alex> My only TRUE wish for tuning performance of Python applications is
    Alex> to have SOME ways to measure memory footprints with sensible
    Alex> guesses about where they come from

Here's a thought.  Debug builds appear to now add a getobjects method to
sys.  Would it be possible to also add another method to sys (also only
available on debug builds) which knows just enough about basic builtin
object types to say a little about how much space it's consuming?  For
example, I could do something like this:

    allocdict = {}
    for o in sys.getobjects(0):
        allocsize = sys.get_object_allocation_size(o)
        # I'm not a fan of {}.setdefault()
        alloc = allocdict.get(type(o), [])
        alloc.append(allocsize) # or alloc.append((allocsize, o))
        allocdict[type(o)] = alloc

Once the list is traversed you can poke around in allocdict figuring out
where your memory went (other than to allocdict itself!).

(I was tempted to suggest another method, but I fear that would just spread
the mess around.  That may also be a viable option though.)

Skip