[ python-Bugs-1722344 ] Thread shutdown exception in Thread.notify()

[SourceForge.net]

Bugs item #1722344, was opened at 2007-05-21 00:24
Message generated for change (Comment added) made by thomasda
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1722344&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Yang Zhang (yangzhang)
Assigned to: Nobody/Anonymous (nobody)
Summary: Thread shutdown exception in Thread.notify()

Initial Comment:
Hi, I sometimes see the following exceptions when shutting down my app (using Python 2.5.1):

  Unhandled exception in thread started by
  Error in sys.excepthook:
  
  Original exception was:
  Exception in thread Thread-3 (most likely raised during interpreter shutdown):
  Traceback (most recent call last):
    File "/usr/local/lib/python2.5/threading.py", line 460, in __bootstrap
    File "/usr/local/lib/python2.5/threading.py", line 440, in run
    File "/home/yang/local/armed/lib/python2.5/site-packages/afx/threads.py", line 71, in worker
    File "/usr/local/lib/python2.5/Queue.py", line 176, in get
    File "/usr/local/lib/python2.5/threading.py", line 248, in notify
  <type 'exceptions.TypeError'>: exceptions must be classes, instances, or strings (deprecated), not NoneType
  Unhandled exception in thread started by
  Error in sys.excepthook:
  
  Original exception was:
  Exception in thread Thread-6 (most likely raised during interpreter shutdown):
  Traceback (most recent call last):
    File "/usr/local/lib/python2.5/threading.py", line 460, in __bootstrap
    File "/usr/local/lib/python2.5/threading.py", line 440, in run
    File "/home/yang/local/armed/lib/python2.5/site-packages/afx/threads.py", line 71, in worker
    File "/usr/local/lib/python2.5/Queue.py", line 176, in get
    File "/usr/local/lib/python2.5/threading.py", line 248, in notify
  <type 'exceptions.TypeError'>: exceptions must be classes, instances, or strings (deprecated), not NoneType
  Unhandled exception in thread started by
  Error in sys.excepthook:
  
  Original exception was:

Here is the code from my application:

        def worker():
            debug( 'starting worker' )
            while True:
                msg = i.get() # <-- THIS IS LINE 71
                if msg is stop_msg: break
                resultbuf, func, args, kwargs = msg
                result, exc = None, None
                try:
                    result = func( *args, **kwargs )
                except:
                    t, v, tb = exc_info()
                    exc = t, v, tb.tb_next
                o.put( ( resultbuf, result, exc ) )
                s.send( 'x' ) # assuming socket.send is thread-safe
            debug( 'stopping worker' )

Here is the origin of the exception (in threading.py):

    def notify(self, n=1):
        assert self._is_owned(), "notify() of un-acquire()d lock" # <-- THIS IS LINE 248
        __waiters = self.__waiters
        waiters = __waiters[:n]
        if not waiters:
            if __debug__:
                self._note("%s.notify(): no waiters", self)
            return
        self._note("%s.notify(): notifying %d waiter%s", self, n,
                   n!=1 and "s" or "")
        for waiter in waiters:
            waiter.release()
            try:
                __waiters.remove(waiter)
            except ValueError:
                pass

I'm not sure why this is happening. The threads are not daemon threads; I terminate them cleanly. When I get a SIGINT (I usu. shut down my app with ctrl-C), I enqueue n stop_msg's to the 'i' Queue so that the n workers can all exit.

Note I usually launch 5 workers, so I'm not consistently getting an exception per worker. Also, I've been unable to reproduce this at will.

----------------------------------------------------------------------

Comment By: Thomas Dybdahl Ahle (thomasda)
Date: 2007-06-06 15:32

Message:
Logged In: YES 
user_id=1304417
Originator: NO

I'm getting the same kind of errors.
I'm using a lot of threads, and one of them throws this thread nearly
everytime I close my program (by gtk.main_quit)

It seems that python sets every variable to None, and then wakeup sleeping
threads, which then crash, as they try to work with the None variables

Exception in thread Thread-3 (most likely raised during interpreter
shutdown):
Traceback (most recent call last):
  File "/usr/lib/python2.4/threading.py", line 442, in __bootstrap
  File
"/home/thomas/Programmering/python/skak/0.7/lib/pychess/System/ThreadPool.py",
line 49, in run
  File "/usr/lib/python2.4/Queue.py", line 89, in put
  File "/usr/lib/python2.4/threading.py", line 237, in notify
exceptions.TypeError: exceptions must be classes, instances, or strings
(deprecated), not NoneType
Unhandled exception in thread started by 
Error in sys.excepthook

----------------------------------------------------------------------

Comment By: Yang Zhang (yangzhang)
Date: 2007-05-21 16:47

Message:
Logged In: YES 
user_id=1207658
Originator: YES

No, as they are not daemon threads.

----------------------------------------------------------------------

Comment By: Gabriel Genellina (gagenellina)
Date: 2007-05-21 14:01

Message:
Logged In: YES 
user_id=479790
Originator: NO

Do you join() the worker threads, waiting until they finish, before
exiting the main thread?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1722344&group_id=5470