[ python-Feature Requests-502236 ] Asynchronous exceptions between threads

[SourceForge.net]

Feature Requests item #502236, was opened at 2002-01-11 03:46
Message generated for change (Comment added) made by akuchling
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=502236&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
>Category: Extension Modules
>Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: paul rubin (phr)
Assigned to: Nobody/Anonymous (nobody)
>Summary: Asynchronous exceptions between threads

Initial Comment:
I may be missing something but there doesn't seem to
be any easy, correct way to exit a Python application
and shut down the interpreter.  sys.exit() raises
the SysExit exception in the current thread but other
threads keep running, so the application doesn't stop.
You can do something brutal like os.kill() your
process, but then cleanup actions (finally clauses)
don't get run.  Or you can create some elaborate
application-specific framework for propagating an
exit flag from one thread to all the rest.  That's
incredibly lame.  

What's needed is a simple function (maybe sys.allexit)
that raises SysExit in all the threads of the
application (it might have to first stop all the 
threads using the GIL or something like that).  

I'm surprised there isn't already something like this, 
but there's been some c.l.py discussion about it and 
it really seems to not be there.  Is there some 
problem with the idea?  I don't see one but I can't be 
sure.

----------------------------------------------------------------------

>Comment By: A.M. Kuchling (akuchling)
Date: 2006-12-21 09:52

Message:
Logged In: YES 
user_id=11375
Originator: NO

Changing summary to be clearer; reclassifying as 'Feature Request' type.


----------------------------------------------------------------------

Comment By: paul rubin (phr)
Date: 2002-01-15 14:40

Message:
Logged In: YES 
user_id=72053

I don't feel qualified to write such a PEP.  You guys are 
more on top of this than I am.  Maybe there's no portable 
way to do asynchronous exceptions between threads.  I don't 
know what happens if you hit ctrl-C when running a multi-
threaded Linux program.  I don't think ctrl-C even works in 
Windows--all you can really do is bring up the task manager 
and blow away the whole process.  But I think asynchronous 
exceptions are worth having even if they're OS specific and 
only work on some systems.

As for how a thread can exit the whole program, I thought 
Guido's first message (saying raise some asynchronous 
exception in all the threads) sounded fine, if it's 
feasible.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-01-15 14:33

Message:
Logged In: YES 
user_id=6380

(1) I think subclassing SystemExit can work.

(2) But the difference is that killing the main thread
without giving it a chance to clean up is worse than killing
another thread.

Does anybody want to write a PEP on thread cancellation? It
sure looks like a hairy issue!

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-01-14 03:40

Message:
Logged In: YES 
user_id=21627

On (1): This is quite unfortunate, as I do think sys.exit
and thread.exit should do different things. There would be
approaches of further subclassing SystemExit; preserving the
property that raising SystemExit in a thread exits the
thread only - would such a solution be acceptable?

On (2): This is no different from raising SystemExit in the
main thread, which also does not invoke finally clauses in
other threads.

I don't think there can be a complete, reliable
implementation of thread cancellation, only a best-effort
approach. For POSIX, I'd suggest sending SIGUSR2 to each
thread through pthread_kill. 

We might need to add a mechanism indicating whether a thread
is ready to be cancelled, similar to the POSIX cancellation
points. That would prevent the signal from arriving in an
arbitrary library function, and defer thread termination
until the library function completes. For blocking system
calls, sending signals would need to be activated
explicitly, to allow aborting them.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-01-12 15:03

Message:
Logged In: YES 
user_id=6380

(1) It's too late to change the meaning of SystemExit --
people are relying on SystemExit exiting the thread only.

(2) If we provide a way for a thread to exit the whole
program, how will finally clauses in other threads
(including the main thread) be executed?

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-01-11 20:05

Message:
Logged In: YES 
user_id=21627

Tim, I'm not surprised that raising SystemExit in a thread
fails to exit Python; the reason simply is that
threadmodule.c:t_bootstrap choses to ignore
PyExc_SystemExit, when it instead should invoke
PyErr_PrintEx instead.

The bug is obviously that exit_thread also raises
SystemExit, when people apparently really want sys.exit and
thread.exit to be two different things. So I think
thread.exit should raise ThreadExit, and raising SystemExit
in a thread should cause Py_Exit to be invoked (along with
all other necessary cleanup). In short, 2.12 of
threadmodule.c was wrong, IMO.


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-01-11 16:16

Message:
Logged In: YES 
user_id=31435

Martin, change your program to do sys.exit(1) inside func() 
on the first iteration (or raise SystemExit similarly), and 
whether you're on Windows or Linux you should find that 
while the func() thread goes away, the main program keeps 
on running.  If you do os._exit(1) instead, on Windows the 
main program does go away, but Guido said it doesn't on 
Linux (which didn't surprise me, given Linux's conflation 
of threads with processes).  See also the "Killing Threads" 
thread in Python-Dev from late May of 2001, which said all 
the same things.

The only thing I may <wink> have learned since then is that 
supposedly the *C* exit() function terminates all threads 
on Linux, via an atexit handler installed by libc, and so 
_exit() on Linux doesn't terminate all threads because _exit
() doesn't call atexit functions.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-01-11 11:19

Message:
Logged In: YES 
user_id=21627

Can you produce an example demonstrating the problem? Please
see the attached thr.py. It terminates fine; I fail to see
the problem.

When SystemExit is raised, it will eventually invoke the C
library's exit(). If that is not incredibly lame, it will
terminate all threads.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-01-11 10:46

Message:
Logged In: YES 
user_id=6380

I'm struggling with this in a threaded distributed app we're
developing for Zope Corp, so I agree that something like
this would be nice. I think for starters, there should be a
way to raise an asynchronous exception in a specific thread;
all threads is easy then.

There are plenty of implementation problems with this,
however: if a thread is blocked in I/O, there's no portable
way to get it to bail out of that I/O operation.

There are also plenty of application-level issues: if
asynchronous exceptions are common, maybe it is necessary to
provide a way to temporarily block such exceptions in order
to provide safety in some situations.

Etc., etc. So please spare us your surprise and help looking
for a mechanism that can be implemented.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=502236&group_id=5470