[Python-bugs-list] [ python-Bugs-754449 ] Exceptions when a thread exits

SourceForge.net noreply@sourceforge.net
Sun, 15 Jun 2003 23:58:51 -0700


Bugs item #754449, was opened at 2003-06-14 03:32
Message generated for change (Comment added) made by bcannon
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=754449&group_id=5470

Category: Threads
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Matthias Klose (doko)
Assigned to: Nobody/Anonymous (nobody)
Summary: Exceptions when a thread exits

Initial Comment:
[forwarded from http://bugs.debian.org/195812]

The application mentioned is offlineimap, available
from ftp://ftp.debian.org/dists/unstable/main/source/.

This behavior is new to Python 2.3.

When my application exits, I get a lot of these messages:

Traceback (most recent call last):
  File "/usr/lib/python2.3/threading.py", line 426, in
__bootstrap
    self.__stop()
  File "/usr/lib/python2.3/threading.py", line 435, in
__stop
    self.__block.notifyAll()
  File "/usr/lib/python2.3/threading.py", line 239, in
notifyAll
    self.notify(len(self.__waiters))
  File "/usr/lib/python2.3/threading.py", line 221, in
notify
    currentThread() # for side-effect
TypeError: 'NoneType' object is not callable
jgoerzen@christoph:~/tree/offlineimap-3.99.18$
./offlineimap.py  -l log -d maildir -a Personal,Excel
Unhandled exception in thread started by <bound method
ExitNotifyThread.__bootstrap of <ExitNotifyThread(Keep
alive LocalExcel, stopped daemon)>>
Traceback (most recent call last):
  File "/usr/lib/python2.3/threading.py", line 426, in
__bootstrap
    self.__stop()
  File "/usr/lib/python2.3/threading.py", line 435, in
__stop
    self.__block.notifyAll()
  File "/usr/lib/python2.3/threading.py", line 239, in
notifyAll
    self.notify(len(self.__waiters))
  File "/usr/lib/python2.3/threading.py", line 221, in
notify
    currentThread() # for side-effect
TypeError: 'NoneType' object is not callable
Unhandled exception in thread started by <bound method
ExitNotifyThread.__bootstrap of <ExitNotifyThread(Keep
alive RemoteExcel, stopped daemon)>>
Traceback (most recent call last):
  File "/usr/lib/python2.3/threading.py", line 426, in
__bootstrap
    self.__stop()
  File "/usr/lib/python2.3/threading.py", line 435, in
__stop
    self.__block.notifyAll()
  File "/usr/lib/python2.3/threading.py", line 239, in
notifyAll
    self.notify(len(self.__waiters))
  File "/usr/lib/python2.3/threading.py", line 221, in
notify
    currentThread() # for side-effect
TypeError: 'NoneType' object is not callable


----------------------------------------------------------------------

>Comment By: Brett Cannon (bcannon)
Date: 2003-06-15 23:58

Message:
Logged In: YES 
user_id=357491

OK, following Tim's advice I checked and it seems that Thread 
calls a method while shutting itself down that calls 
Condition.notifyAll which calls currentThread which is a global.  It 
would appear that offlineimap is leaving its threads running, the 
program gets shut down, the threads raise an error while 
shutting down (probably because things are being torn down), 
this triggers the stopping method in Thread, and this raises its 
own exception because of the teardown which is what we are 
seeing as the TypeError.

So the question is whether Condition should store a local 
reference to currentThread or not.  It is not the most pure 
solution since this shutdown issue is not Condition's, but the only 
other solution I can think of is to have Thread keep a reference 
to currentThread, inject it into the current frame's global 
namespace, call Condition.notifyAll, and then remove the 
reference from the frame again.  I vote for the cleaner, less pure 
solution.  =)

Am I insane on this or does it at least sound like this is the 
problem and proper solution?

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2003-06-15 23:19

Message:
Logged In: YES 
user_id=357491

Nuts.  For some reason I thought the OP had said when threads 
were exiting.

I will ask on python-dev for a good explanation of what happens 
when Python is shutting down.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-06-15 18:05

Message:
Logged In: YES 
user_id=31435

Note that the OP said "when my application exits".  When 
Python is tearing down the universe, it systematically sets 
module-global bindings to None.  currentThread() is a module 
global.  I can't make more time for this now, so for more info 
talk about it on Python-Dev.

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2003-06-15 16:44

Message:
Logged In: YES 
user_id=357491

Well, I'm stumped.  I checked the diff from when 2.2 was initially 
released until now and the only change that seems to be related 
to any of this is that what is returned by currentThread is not 
saved in a variable.  But since the error is the calling of 
currentThread itself and not saving the return value I don't see 
how that would affect anything.

I also went through offlineimap but I didn't see anything there 
that seemed to be setting currentThread to None.  Although 
since several files do ``import *`` so there still is a possibility of 
overwriting currentThread locally.

So, for my owning learning and to help solve this, I have written 
a tracing function that writes to stderr using the logging package 
when it detects that either currentThread or 
threading.currentThread has been set to None, locally or globally 
(I assume the code is not injecting into builtins so I didn't bother 
checking there).  The file is named tracer.py and I have attached 
it to this bug report.  If you can execute 
``sys.settrace(tracer.trace_currentThread)`` before offlinemap 
starts executing and immediately within each thread (it has to be 
called in *every* thread since tracing functions are no inherited 
from the main thread) it should print out a message when 
currentThread becomes None.  If you *really* want to make this 
robust you can also have it check sys.modules['threading'] every 
time as well, but I figure there is not going to be much renaming 
and masking of currentThread.

----------------------------------------------------------------------

Comment By: Matthias Klose (doko)
Date: 2003-06-15 01:44

Message:
Logged In: YES 
user_id=60903

Please see
http://packages.debian.org/unstable/mail/offlineimap.html

or for the tarball:
http://ftp.debian.org/debian/pool/main/o/offlineimap/offlineimap_3.99.18.tar.gz

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2003-06-15 01:19

Message:
Logged In: YES 
user_id=357491

I went to the FTP site and all I found was some huge, 
compressed files (after changing the path to ftp://ftp.debian.org/
debian/dists/sid/main/source); no specific program called 
offlinemap.  If it is in one of those files can you just add the file 
to the bug report?

As for the reported bug, it looks like currentThread is being 
redefined, although how that is happening is beyond me.  I 
checked the 'threading' module and no where is currentThread 
redefined which could lead to None.  Has the app being run been 
changed at all since Python 2.2 was released?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=754449&group_id=5470