Threading problems at program exit
Dave Cole
djc at object-craft.com.au
Sat Nov 30 17:36:07 EST 2002
Dave> The following program demonstrates a problem I am experiencing with
Dave> the threading module. I have an object (A) which holds a lock on
Dave> another object (B). When object A is deleted I want it to release any
Dave> lock it may still be holding on object B.
Dave> Everything works fine except when the program terminates.
Dave> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Dave> import sys
Dave> import threading
Dave> class Locked:
Dave> def __init__(self, lock):
Dave> self._lock = lock
Dave> self._lock_count = 0
Dave> self._thread = None
Dave> self._log = sys.stderr.write
Dave> self._current_thread = threading.currentThread
Dave> self.lock()
Dave> def lock(self):
Dave> self._log('locked in %s\n' % self._current_thread())
Dave> self._lock.acquire()
Dave> self._lock_count += 1
Dave> def unlock(self):
Dave> self._log('unlocked in %s\n' % self._current_thread())
Dave> self._lock_count -= 1
Dave> self._lock.release()
Dave> def __del__(self):
Dave> while self._lock_count:
Dave> self.unlock()
Dave> obj = Locked(threading.RLock())
Dave> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Dave> When I run this program I get the following:
Dave> locked in <_MainThread(MainThread, started)>
Dave> unlocked in <_DummyThread(Dummy-1, started daemon)>
Dave> Exception exceptions.AssertionError: <exceptions.AssertionError instance at 0x814fbfc> in <bound method Locked.__del__ of <__main__.Locked instance at 0x816e5f4>> ignored
Dave> It looks like interpreter is deleting thread objects before objects
Dave> which hold locks in those threads. Is there any kosher way I can
Dave> avoid the problem?
Dave> The only way I can think to fix it is a bit non-kosher:
Dave> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Dave> import sys
Dave> import threading
Dave> class Locked:
Dave> def __init__(self, lock):
Dave> self._lock = lock
Dave> self._lock_count = 0
Dave> self._thread = None
Dave> self._log = sys.stderr.write
Dave> self._current_thread = threading.currentThread
Dave> self.lock()
Dave> def lock(self):
Dave> self._log('locked in %s\n' % self._current_thread())
Dave> self._lock.acquire()
Dave> self._lock_count += 1
Dave> def unlock(self):
Dave> self._log('unlocked in %s\n' % self._current_thread())
Dave> self._lock_count -= 1
Dave> self._lock.release()
Dave> def __del__(self):
Dave> if self._lock_count:
Dave> count, owner = self._lock._release_save()
Dave> self._log('owner was %s\n' % owner)
Dave> owner = self._current_thread()
Dave> self._log('owner is now %s\n' % owner)
Dave> self._lock._acquire_restore((count, owner))
Dave> while self._lock_count:
Dave> self.unlock()
Dave> obj = Locked(threading.RLock())
Dave> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Dave> This prints the following:
Dave> locked in <_MainThread(MainThread, started)>
Dave> owner was <_MainThread(MainThread, stopped)>
Dave> owner is now <_DummyThread(Dummy-1, started daemon)>
Dave> unlocked in <_DummyThread(Dummy-1, started daemon)>
Dave> The trouble is that it requires the use of private methods of the
Dave> RLock class.
Dave> Is there a better or more kosher way?
Ype> You might consider not using the __del__ method at all because
Ype> 'it is not guaranteed that __del__() methods are called for
Ype> objects that still exist when the interpreter exits. ' (quoted
Ype> from the language ref). Then use:
Ype>
Ype> obj = Locked(threading.RLock())
Ype> try:
Ype> # whatever needs to be done with obj
Ype> finally:
Ype> obj.unlock()
That won't work in this case because the resource needs to be locked
over a span of client code.
I should be more specific...
The code in question is my Python bindings for Sybase. According the
the DB-API specification a database module must provide Connection
objects for managing database connections, and Cursor objects for
executing commands over a Connection. The DB-API implies that you
should be able to do something like the following:
db = Sybase.connect(...) # return Connection
# in thread1
c1 = db.cursor()
c1.execute('select * blah')
while 1:
row = c1.fetchone()
if not row:
break
# in thread2
c2 = db.cursor()
c2.execute('select * from blahblah')
while 1:
row = c2.fetchone()
if not row:
break
Now the problem is that even though the database connection can be
shared between threads it can only support a single query in flight at
a time. This means that while a cursor is fetching results over a
Connection I need the Cursor to maintain a lock on the Connection. I
do not have any control over the structure of the client code which is
using the Cursor.
In my code the Cursor obtains a lock on the Connection at the start of
a result set and releases the lock at the end of the result. This has
allowed one user to implement a multi-threaded server which
transparently shares a limited number of database connections between
a much larger set of client handling threads.
The __del__ requirement comes in because the DB-API does not require
that you complete result fetching with a cursor. It also does not
require that you close a cursor. This means that the following is a
perfectly fine program:
db = Sybase.connect(...)
c = db.cursor()
c.execute('select * reallbigtable')
row = c.fetchone()
Client code does not need to perform any cleanup.
- Dave
--
http://www.object-craft.com.au
More information about the Python-list
mailing list