[Python-bugs-list] [ python-Bugs-704180 ] Memory leak: threading.Thread.__init__(self, target=self.xx)
SourceForge.net
noreply@sourceforge.net
Sun, 23 Mar 2003 05:52:26 -0800
Bugs item #704180, was opened at 2003-03-15 16:44
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=704180&group_id=5470
Category: Threads
Group: Python 2.2.2
Status: Closed
Resolution: Invalid
Priority: 5
Submitted By: Darrell Gallion (dgallion)
Assigned to: Nobody/Anonymous (nobody)
Summary: Memory leak: threading.Thread.__init__(self, target=self.xx)
Initial Comment:
This call
threading.Thread.__init__(self, target=self.xxx)
makes this assignment
self.__target = target
Which looks like a circular refrence to me.
weakref anyone ?
The following test code shows the leak in action.
--Darrell
import threading, time
class T1(threading.Thread):
def __init__(self, leak=0):
self.leak=leak
if leak:
threading.Thread.__init__(self, target=self.m1)
else:
threading.Thread.__init__(self)
self.start()
def run(self):
pass
def m1(self):
pass
def __del__(self):
print 'DEL:', self, "leak:",self.leak
for x in range(10):
T1(leak=1)
for x in range(10):
T1()
time.sleep(1)
----------------------------------------------------------------------
>Comment By: Darrell Gallion (dgallion)
Date: 2003-03-23 13:52
Message:
Logged In: YES
user_id=68151
My problem was solved by moving the thread options into a group of
functions and passing one of those as the target. These threads have a
shutDown method that didn't work before because of this target problem.
This isn't a problem for me now, yet it seemed like it could have been
avoided in the first place.
Thanks
----------------------------------------------------------------------
Comment By: Tim Peters (tim_one)
Date: 2003-03-22 22:45
Message:
Logged In: YES
user_id=31435
As Martin said, a weakref won't solve anything here.
I remain unclear on what the real problem is. You said that
the timing of when an object gets destroyed is important to
your app, but you have to realize that Python makes no
guarantees about object destruction time. If your thread is
holding on to some critical resource that must be freed in a
timely manner, the only sane way to proceed is to give your
thread a release() or close() method that's called explicitly
to free the critical resource. That should never be left to
the vagaries of garbage collection (whether via refcount or
the gc module).
BTW, I don't believe I've ever seen code that passed a
bound method of a Thread subclass to Thread.__init__
before. The target argument was meant to ease migrating
old code that used the thread module (where specifying a
target function was necessary). Code written with
threading.py in mind invariably (IME) overrides the run()
method instead (which is the intent of the design).
----------------------------------------------------------------------
Comment By: Martin v. Löwis (loewis)
Date: 2003-03-22 21:30
Message:
Logged In: YES
user_id=21627
That won't work. In your example, the method object is
created with the expression self.m1, and the only reference
to it is the target parameter. So if Thread would keep just
a weak reference to target, then the method object would go
away just after __init__ completes. As the result, the
Thread's .run won't find the target.
----------------------------------------------------------------------
Comment By: Darrell Gallion (dgallion)
Date: 2003-03-22 20:36
Message:
Logged In: YES
user_id=68151
Using a threading.Thread as a base class invites you to pass target methods
other than run. Which sets up the cycle. It would be nice if the target could
be a weakref or was converted to a weakref by Thread.
This simple change would prevent others from wasting time on this problem.
----------------------------------------------------------------------
Comment By: Martin v. Löwis (loewis)
Date: 2003-03-21 18:22
Message:
Logged In: YES
user_id=21627
I fail to see a bug in Python here. You are passing a
reference to an object to the __init__ of the base class, so
you should know you are creating cycles.
I recommend to re-implement T1.run instead of using the
target= constructor argument.
----------------------------------------------------------------------
Comment By: Darrell Gallion (dgallion)
Date: 2003-03-16 16:45
Message:
Logged In: YES
user_id=68151
I know about the __del__ issue with gc, and your right this example worked
fine with it removed. The much larger app that sent me on this chase wasn't
fixed by the __del__ method. Yet it was fixed by moving the target method
to a function call instead of a method on the object.
It's a large app and the timing of when an object is destroyed matters.
The following helps me see my problem.
import threading, time, weakref
class T1(threading.Thread):
def __init__(self):
self.go=1
threading.Thread.__init__(self, target=self.m1)
self.start()
def m1(self):
while self.go:
time.sleep(0.0001)
objs={}
while 1:
t1 = weakref.ref(T1())
objs[t1]=1
t1().go=0
time.sleep(0.01)
for o in objs.keys():
if o() == None:
del objs[o]
print len(objs),
----------------------------------------------------------------------
Comment By: Tim Peters (tim_one)
Date: 2003-03-16 04:39
Message:
Logged In: YES
user_id=31435
The "leak" is due to that you gave this class a __del__
method: cyclic garbage collection won't magically reclaim
objects in cycles with __del__ methods. Remove the
__del__ and you can run this all day without the process
size growing. If you leave the __del__ in, you can find all
your uncollectible thread objects in gc.garbage (and use
that to break the cycles and get them collected, if you like --
I'd recommend not using __del__ at all, though).
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=704180&group_id=5470