Where'd my memory go? (was Re: [Python-Dev] Dictionary sparseness)
Skip Montanaro
skip@pobox.com
Mon, 5 May 2003 14:35:23 -0500
Alex> My only TRUE wish for tuning performance of Python applications is
Alex> to have SOME ways to measure memory footprints with sensible
Alex> guesses about where they come from
Here's a thought. Debug builds appear to now add a getobjects method to
sys. Would it be possible to also add another method to sys (also only
available on debug builds) which knows just enough about basic builtin
object types to say a little about how much space it's consuming? For
example, I could do something like this:
allocdict = {}
for o in sys.getobjects(0):
allocsize = sys.get_object_allocation_size(o)
# I'm not a fan of {}.setdefault()
alloc = allocdict.get(type(o), [])
alloc.append(allocsize) # or alloc.append((allocsize, o))
allocdict[type(o)] = alloc
Once the list is traversed you can poke around in allocdict figuring out
where your memory went (other than to allocdict itself!).
(I was tempted to suggest another method, but I fear that would just spread
the mess around. That may also be a viable option though.)
Skip