memory leak with dynamically defined functions?

[Duncan Booth]

Harald Kirsch <kirschh at lionbioscience.com> wrote in
news:yv2k818t3wk.fsf at lionsp093.lion-ag.de: 

>> > >>> def silliest_func():
>> > >>>     x = [0] * (2**10)
>> > >>>     def inner_silliest_func(x=x): pass
>> > >>>     return inner_silliest_func
>> > >>> 
>> > >>> blarg = {}
>> > >>> for i in range(2**13): blarg[i] = silliest_func()
>> 
>> This creates 2**13 references to one and the same object, namely
>> silliest_func which (I suspect) has just one reference to an object
>> inner_silliest_func. 
> 
> Should read: 
> This creates 2**13 references to one and the same object, namely
> inner_silliest_func.

Your understanding seems to be slightly flawed here. This actually
creates 1 reference to each of 2**13 objects. Each object is a function
called inner_silliest_func with a default argument, but every time you
call silliest_func you create a new object 'x' and a new function using 
that x as an argument.

> So it appears to me that inner functions and/or their default arguments
> are not getting properly collected in CPython.  If this is the case, it
> poses quite a problem for Mojo Nation, as we have relied heavily on the
> convenient closure- like feature of passing around references to
> dynamically defined functions with default arguments.

Is there a good reason why you cannot upgrade to Python 2.1? It appears 
that this problem is fixed in 2.1 as the following example shows:

--- begin refs.py ---
import weakref
class Tracker:
    def __init__(self):
        self.__trackedobjects = weakref.WeakValueDictionary()
        
    def trackLifetime(self, object, name=None):
        """Track the lifetime of the object. Adds it to a weak reference 
dictionary.
        At the end of the test the weak dictionary must be empty again."""

        name = name or repr(object)
        tracked = self.__trackedobjects
        tracked[(name, id(object))] = object

    def printStatus(self):
        print "Tracker is tracking %d objects" % len(self.__trackedobjects)

    def verifyTrackedObjects(self):
        tracked = self.__trackedobjects

        if len(tracked) != 0:
            names = []
            for (name, id) in tracked.keys():
                names.append(name)
            msg = "Objects not freed: " + ", ".join(names)
            assert len(tracked)==0, msg

tracker = Tracker()
tracker.printStatus()

def silliest_func():
     x = [0]
     def inner_silliest_func(x=x): pass
     tracker.trackLifetime(inner_silliest_func)
     return inner_silliest_func
 
blarg = {}
for i in range(10): blarg[i] = silliest_func()
tracker.printStatus()

del blarg
tracker.printStatus()
tracker.verifyTrackedObjects()
--- end ---
Which gives you the output:
Tracker is tracking 0 objects
Tracker is tracking 10 objects
Tracker is tracking 0 objects

If you comment out the line 'del blarg' you will get a list of unfreed 
objects.

-- 
Duncan Booth                                             duncan at rcp.co.uk
int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
"\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?