[Python-bugs-list] [ python-Bugs-704180 ] Memory leak: threading.Thread.__init__(self, target=self.xx)

[SourceForge.net]

Bugs item #704180, was opened at 2003-03-15 17:44
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=704180&group_id=5470

Category: Threads
Group: Python 2.2.2
Status: Closed
Resolution: Invalid
Priority: 5
Submitted By: Darrell Gallion (dgallion)
Assigned to: Nobody/Anonymous (nobody)
Summary: Memory leak: threading.Thread.__init__(self, target=self.xx)

Initial Comment:
This call  
	threading.Thread.__init__(self, target=self.xxx) 
makes this assignment  
	self.__target = target 
Which looks like a circular refrence to me.  
weakref anyone ? 
 
The following test code shows the leak in action. 
 
--Darrell 
 
 
import threading, time 
 
class T1(threading.Thread): 
    def __init__(self, leak=0): 
        self.leak=leak 
        if leak: 
            threading.Thread.__init__(self, target=self.m1) 
        else: 
            threading.Thread.__init__(self) 
        self.start() 
         
    def run(self): 
        pass 
         
    def m1(self): 
        pass 
         
    def __del__(self): 
        print 'DEL:', self, "leak:",self.leak 
         
 
for x in range(10): 
    T1(leak=1) 
for x in range(10): 
    T1() 
time.sleep(1) 
 
 

----------------------------------------------------------------------

>Comment By: Martin v. Löwis (loewis)
Date: 2003-03-22 22:30

Message:
Logged In: YES 
user_id=21627

That won't work. In your example, the method object is
created with the expression self.m1, and the only reference
to it is the target parameter. So if Thread would keep just
a weak reference to target, then the method object would go
away just after __init__ completes. As the result, the
Thread's .run won't find the target.


----------------------------------------------------------------------

Comment By: Darrell Gallion (dgallion)
Date: 2003-03-22 21:36

Message:
Logged In: YES 
user_id=68151

Using a threading.Thread as a base class invites you to pass target methods 
other than run. Which sets up the cycle. It would be nice if the target could 
be a weakref or was converted to a weakref by Thread. 
 
This simple change would prevent others from wasting time on this problem. 
 

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2003-03-21 19:22

Message:
Logged In: YES 
user_id=21627

I fail to see a bug in Python here. You are passing a
reference to an object to the __init__ of the base class, so
you should know you are creating cycles.

I recommend to re-implement T1.run instead of using the
target= constructor argument.

----------------------------------------------------------------------

Comment By: Darrell Gallion (dgallion)
Date: 2003-03-16 17:45

Message:
Logged In: YES 
user_id=68151

I know about the __del__ issue with gc, and your right this example worked 
fine with it removed. The much larger app that sent me on this chase wasn't 
fixed by the __del__ method. Yet it was fixed by moving the target method 
to a function call instead of a method on the object. 
 
It's a large app and the timing of when an object is destroyed matters. 
The following helps me see my problem.  
 
import threading, time, weakref 
 
class T1(threading.Thread): 
    def __init__(self): 
        self.go=1 
        threading.Thread.__init__(self, target=self.m1) 
        self.start() 
         
    def m1(self): 
        while self.go: 
            time.sleep(0.0001) 
         
 
objs={} 
while 1: 
    t1 = weakref.ref(T1()) 
    objs[t1]=1 
    t1().go=0 
    time.sleep(0.01) 
    for o in objs.keys(): 
        if o() == None: 
            del objs[o] 
    print len(objs), 
 
 
         
 
 
 

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-03-16 05:39

Message:
Logged In: YES 
user_id=31435

The "leak" is due to that you gave this class a __del__ 
method:  cyclic garbage collection won't magically reclaim 
objects in cycles with __del__ methods.  Remove the 
__del__ and you can run this all day without the process 
size growing.  If you leave the __del__ in, you can find all 
your uncollectible thread objects in gc.garbage (and use 
that to break the cycles and get them collected, if you like --
 I'd recommend not using __del__ at all, though).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=704180&group_id=5470