[Python-bugs-list] [ python-Bugs-544473 ] Queue module can deadlock
noreply@sourceforge.net
noreply@sourceforge.net
Thu, 18 Apr 2002 17:12:51 -0700
Bugs item #544473, was opened at 2002-04-16 12:09
You can respond by visiting:
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=544473&group_id=5470
Category: Python Library
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Mark Hammond (mhammond)
Assigned to: Mark Hammond (mhammond)
Summary: Queue module can deadlock
Initial Comment:
Prompted by a users experimentation where he was able
to show Pythonwin hanging, I have found a case where
the Queue module deadlocks. It appears that if a
RuntimeError is generated while queueing data, the
queue lock will remain held.
Attaching a test script. This hangs on Win2k and
Linux. Win2k callstack shows Queue is waiting for the
following lock (line 53)
if block:
self.fsema.acquire()
Presumably (but untested at the moment), the previous
call to q.put(), which triggered a "RuntimeError:
maximum recursion depth exceeded", left the lock acquired.
It seems a few choice try/finally statements in the
Queue module would go along way :) I will try and get
to this soon.
----------------------------------------------------------------------
>Comment By: Mark Hammond (mhammond)
Date: 2002-04-19 10:12
Message:
Logged In: YES
user_id=14198
Checking in Queue.py;
/cvsroot/python/python/dist/src/Lib/Queue.py,v <-- Queue.py
new revision: 1.15; previous revision: 1.14
Checking in test/test_queue.py;
/cvsroot/python/python/dist/src/Lib/test/test_queue.py,v
<-- test_queue.py
initial revision: 1.1
bugfix candidate.
----------------------------------------------------------------------
Comment By: Tim Peters (tim_one)
Date: 2002-04-19 04:20
Message:
Logged In: YES
user_id=31435
Changed to Accepted to encourage you to check it in <wink>.
Note that expected-output files are not *required* for
tests anymore, and indeed are discouraged. Better to print
purely informational msgs under a
if test_support.verbose:
print "yadda, yadda"
guard, and not create any expected-output file.
In the checkin msg(s), be sure to say this is a "bugfix
candidate" patch.
----------------------------------------------------------------------
Comment By: Mark Hammond (mhammond)
Date: 2002-04-18 21:30
Message:
Logged In: YES
user_id=14198
Good idea re subclassing to provoke the deadlock! Attaching
the exact same Queue.py patch, plus a reasonable test case -
both blocking and non-blocking gets and puts are tested in
the succeeding and failing cases. Without the Queue.py
patch, both Linux and Windows deadlock in the tests. With
the patch, both succeed. If no objections I will check it in.
----------------------------------------------------------------------
Comment By: Tim Peters (tim_one)
Date: 2002-04-18 15:09
Message:
Logged In: YES
user_id=31435
The patch looks good to me, but I would like to see a test
case. Note that you can build a reliable test case by
subclassing Queue and overriding its _put and/or _get
methods, arranging to raise an exception deliberately from
them. Any exception in these would leave a semaphore and
self.mutex held forever in the current Queue.py.
----------------------------------------------------------------------
Comment By: Mark Hammond (mhammond)
Date: 2002-04-16 22:18
Message:
Logged In: YES
user_id=14198
This patch seems reasonable to me. Tim, a quick look would
be appreciated. Writing a reasonable test case seems
difficult :( It doesn't seem worth adding this little test
script to the test suite.
----------------------------------------------------------------------
Comment By: Tim Peters (tim_one)
Date: 2002-04-16 12:47
Message:
Logged In: YES
user_id=31435
Yup, it hangs on Win98 too. Nice job of whittling it down,
Mark! We've been looking for an excuse to make Queue
slower too <wink>.
----------------------------------------------------------------------
You can respond by visiting:
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=544473&group_id=5470