[Python-bugs-list] [ python-Bugs-621548 ] Numerous defunct threads left behind

[SourceForge.net]

Bugs item #621548, was opened at 2002-10-10 15:25
Message generated for change (Comment added) made by jamincollins
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=621548&group_id=5470

Category: Threads
Group: Python 2.3
Status: Open
Resolution: None
Priority: 3
Submitted By: Jamin W. Collins (jamincollins)
Assigned to: Nobody/Anonymous (nobody)
Summary: Numerous defunct threads left behind

Initial Comment:
I originally noticed this problem with Debian's
OfflineIMAP package and filed a report there:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=162369

The short of it is that when running offlineimap for an
extended period of time it will eventually leave a
large number of defunct threads behind that eventually
use up all available system processes.  Killing the
original offlineimap thread clears the problem for a while.

The Debian maintainer of OfflineIMAP referred this over
to the python maintainer, who in turn has asked (as you
can see from the link above) that I file a report here.

If there is any more information I can provide (beyond
that in the Debian case already) please let me know.

----------------------------------------------------------------------

>Comment By: Jamin W. Collins (jamincollins)
Date: 2003-04-17 19:55

Message:
Logged In: YES 
user_id=88136

Go ahead and close it.  Problem still exists.  I don't know
enough about either to say which is at fault.  Nor do I know
enough about python to attempt to replicate it.  I only know
the the OfflineIMAP folks say it's not them and it would
seem now it's not Python either.

*shrug* I tried.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-04-17 19:27

Message:
Logged In: YES 
user_id=6380

I don't understand aything that is being said in the last
two comments. If anything, the bug appears to be in the
application (with which I am not familiar).

I'm going to close soon this unless someone has a bug they
can demonstrate with a small sample program that does not
involve OfflineIMAP.


----------------------------------------------------------------------

Comment By: Jamin W. Collins (jamincollins)
Date: 2003-04-17 17:41

Message:
Logged In: YES 
user_id=88136

I used both the TK and the TTY interfaces.  In both cases
(using both Python 2.2 and 2.3) zombie threads would
accumlate over time when it was left running in daemon mode.

----------------------------------------------------------------------

Comment By: John Goerzen (jgoerzen)
Date: 2003-04-17 14:27

Message:
Logged In: YES 
user_id=491567

OfflineIMAP has only two cases where one of these things 
might happen: 
 
1) in the Curses interface module, where it does a fork at the 
very beginning to check to see if Curses will work (just a quick 
fork/exit/cleanup) 
 
2) with the preauth code, where it does a popen2 to a local 
imap daemon. 
 
Few people use preauth, and the Curses problem can easily 
be dismissed by switching to another ui (TTY.TTYUI perhaps?) 
 
I have observed that even the TTY.TTYUI leaves processes 
hanging around in Python 2.2 but not in Python 2.3. 
 
I suspect the original submitter is running the Tk interface; my 
own experimentation has shown it leaves lots of processes 
hanging around. 
 
However -- I have never seen zombie processes hanging 
around. 
 
The processes I see have blocked themselves with 
rt_sigsuspend. 

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-10-14 19:35

Message:
Logged In: YES 
user_id=6380

I agree with Martin von Loewis - this does not appear to be
a Python bug.

When a Python thread exits, it does not show up in the ps
listing as <defunct>; that to me suggests that there *is*
some forking going on, perhaps under the guise of system()
or popen(); or perhaps there's a bug in the C library's
thread support.

Lowering the priority as a result. I'll try to followup in
the Debian thread as well, if there's a web interface.


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-10-13 06:05

Message:
Logged In: YES 
user_id=21627

The analysis that this is a Python bug is most likely wrong.
You write

> a number of defunct/zombie processes are spawned.

then John Goerzen writes

> It is likely, in any case, that this is a bug in Python
itself.   The 
> OfflineIMAP program does not fork 

However, John later points out that there are many threads
used in this package. Please understand that threads show up
as processes in Linux ps(1).

IOW, what you are seeing are the many threads, not any
additional processes.

It appears that the system is waiting for somebody to call
pthread_join. This should not be necessary, since Python
invokes pthread_detach for all of its threads. So if the
system does not reclaim terminated threads, it seems there
is a bug in your C library.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=621548&group_id=5470