[Python-bugs-list] [ python-Bugs-621548 ] Numerous defunct threads left behind

SourceForge.net noreply@sourceforge.net
Thu, 17 Apr 2003 12:27:02 -0700


Bugs item #621548, was opened at 2002-10-10 15:25
Message generated for change (Comment added) made by jgoerzen
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=621548&group_id=5470

Category: Threads
Group: Python 2.3
Status: Open
Resolution: None
Priority: 3
Submitted By: Jamin W. Collins (jamincollins)
Assigned to: Nobody/Anonymous (nobody)
Summary: Numerous defunct threads left behind

Initial Comment:
I originally noticed this problem with Debian's
OfflineIMAP package and filed a report there:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=162369

The short of it is that when running offlineimap for an
extended period of time it will eventually leave a
large number of defunct threads behind that eventually
use up all available system processes.  Killing the
original offlineimap thread clears the problem for a while.

The Debian maintainer of OfflineIMAP referred this over
to the python maintainer, who in turn has asked (as you
can see from the link above) that I file a report here.

If there is any more information I can provide (beyond
that in the Debian case already) please let me know.

----------------------------------------------------------------------

Comment By: John Goerzen (jgoerzen)
Date: 2003-04-17 14:27

Message:
Logged In: YES 
user_id=491567

OfflineIMAP has only two cases where one of these things 
might happen: 
 
1) in the Curses interface module, where it does a fork at the 
very beginning to check to see if Curses will work (just a quick 
fork/exit/cleanup) 
 
2) with the preauth code, where it does a popen2 to a local 
imap daemon. 
 
Few people use preauth, and the Curses problem can easily 
be dismissed by switching to another ui (TTY.TTYUI perhaps?) 
 
I have observed that even the TTY.TTYUI leaves processes 
hanging around in Python 2.2 but not in Python 2.3. 
 
I suspect the original submitter is running the Tk interface; my 
own experimentation has shown it leaves lots of processes 
hanging around. 
 
However -- I have never seen zombie processes hanging 
around. 
 
The processes I see have blocked themselves with 
rt_sigsuspend. 

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-10-14 19:35

Message:
Logged In: YES 
user_id=6380

I agree with Martin von Loewis - this does not appear to be
a Python bug.

When a Python thread exits, it does not show up in the ps
listing as <defunct>; that to me suggests that there *is*
some forking going on, perhaps under the guise of system()
or popen(); or perhaps there's a bug in the C library's
thread support.

Lowering the priority as a result. I'll try to followup in
the Debian thread as well, if there's a web interface.


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-10-13 06:05

Message:
Logged In: YES 
user_id=21627

The analysis that this is a Python bug is most likely wrong.
You write

> a number of defunct/zombie processes are spawned.

then John Goerzen writes

> It is likely, in any case, that this is a bug in Python
itself.   The 
> OfflineIMAP program does not fork 

However, John later points out that there are many threads
used in this package. Please understand that threads show up
as processes in Linux ps(1).

IOW, what you are seeing are the many threads, not any
additional processes.

It appears that the system is waiting for somebody to call
pthread_join. This should not be necessary, since Python
invokes pthread_detach for all of its threads. So if the
system does not reclaim terminated threads, it seems there
is a bug in your C library.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=621548&group_id=5470