Memory leak/gc.get_objects()/Improved gc in version 2.5

crazy420fingers at gmail.com crazy420fingers at gmail.com
Tue Oct 9 02:44:03 CEST 2007


I'm running a python program that simulates a wireless network
protocol for a certain number of "frames" (measure of time).  I've
observed the following:

1. The memory consumption of the program grows as the number of frames
I simulate increases.

To verify this, I've used two methods, which I invoke after every
frame simulated:

--  Parsing the /proc/<pid>/status file as in:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/286222
--  Using ps vg | grep python | awk '!/grep/ {print " ",$8}'  in an
os.system() call.

The memory usage vs. frame number graph shows some big "jumps" at
certain points, and, after a large number of frames, shows a steady
upward slope

2. I think I've verified that the objects I instantiate are actually
freed-- I'm therefore assuming that this "leak" is "caused" by
python's garbage collection mechanism. I count the number of objects I
generate that are being tracked by gc as follows:

    gc.collect()
    objCount = {}
    objList = gc.get_objects()
    for obj in objList:
        if getattr(obj, "__class__", None):
            name = obj.__class__.__name__
            if objCount.has_key(name):
                objCount[name] += 1
            else:
                objCount[name] = 1

    for name in objCount:
        print name, " :", objCount[name]

   del objList

Running this snippet every hundred frames or so, shows that the number
of objects managed by gc is not growing.

I upgraded to Python 2.5. in an attempt to solve this problem. The
only change in my observations from version 2.4 is that the absolute
memory usage level seems to have dropped. However, I still see the
jumps in memory usage at the same points in time.

Can anybody explain why the memory usage shows significant jumps (~200
kB or ~500 kb) over time (i.e. "frames") even though there is no
apparent increase in the objects managed by gc? Note that I'm calling
gc.collect() regularly.

Thanks for your attention,

Arvind




More information about the Python-list mailing list