Hunting a memory leak
mwh at python.net
Fri Aug 29 14:10:03 CEST 2003
Debian User <falted at inspiron.openlc.org> writes:
> I'm trying to discover a memory leak on a program of mine. I've taken
> several approaches, but the leak still resists to appear.
> First of all, I've tried to use the garbage collector to look for
> uncollectable objects.
> Then I've taken an approach that I've seen in python developers list
> contributed by Walter Dörwald, that basically consists in creating a
> debug version of python, create a unitest with the leaking code, and
> modify the unittest.py to extract the increment of total reference
> counting in that code (see
Well, somewhere in that same thread are various references to a
TrackRefs class. Have you tried using that? It should tell you what
type of object is leaking, which is a good start.
> With that, I see that my reference count grows by one each time the
> test execute. But the problem is: is there some way to look at the
> object (or make a memory dump) that is leaking?.
See above :-)
> I've used valgrind (http://developer.kde.org/~sewardj/) to see if it
> could detect the leak. In fact, it detects a bunch of them, but I am
> afraid that they are not related with the leak I'm looking for. I am
> saying that because, when I loop over my leaky code, valgrind always
> report the same amount of leaky memory, independently of the number of
> iterations (while top is telling me that memory use is growing!).
There are various things (interned strings, f'ex) that always tend to
be alive at the end of a Python program: these are only leaks in a
very warped sense.
I don't know if there's a way to get vaglrind to tell you what's
allocated but not deallocated between two arbitrary points of program
> My code uses extension modules in C, so I am afraid this does not
> contribute to alleviate the problem.
Well, in all likelyhood, the bug is IN the C extension module. Have
you tried stepping through the code in a debugger? Sometime's that's
a good way of spotting a logic error.
> I am sorry, but I cannot be more explicit about the code because it
> is quite complex (it is the PyTables package, http://pytables.sf.net),
> and I was unable to make a simple example to be published
> here. However, if anyone is tempted to have a look at the code, you
> can download it from
> (http://sourceforge.net/project/showfiles.php?group_id=63486). I am
> attaching a unittest that exposes the leak.
> I am a bit desperate. Any hint?
Not really. Try using TrackRefs.
I'm about to search Google for contract assassins to go to Iomega
and HP's programming groups and kill everyone there with some kind
of electrically charged rusty barbed thing.
-- http://bofhcam.org/journal/journal.html, 2002-01-08
More information about the Python-list