Hi All, Currently it is not possible to "kill" thread which is blocked. The rationale for this is handling situations when thread is blocked - e.g. when thread is quering DB when lock occurred on Database. In this case, the main thread has no way how to stop the blocked thread. Killing a thread is also popular question - see [1][2]. pthread library and Windows API contains mechanisms for forced termination of threads - see [3] and [4]. It is also simple to use them using ctypes library but after this action one need to "clean" internal data structures which is bad practice: import time import threading import ctypes def handler(): # blocked thread handler time.sleep(1000) t = threading.Thread(name='bar', target=handler) libpt = ctypes.cdll.LoadLibrary("libpthread.so.0") t.start() libpt.pthread_cancel(ctypes.c_ulong(t.ident)) # This is nasty cleaning of internal python structures del threading._active[t.ident] Is if feasible to add canceling threads to python threading library? I am willing to help creating a patch (at least for linux). [1] https://stackoverflow.com/questions/323972/is-there-any-way-to-kill-a-thread [2] https://www.geeksforgeeks.org/python-different-ways-to-kill-a-thread/ [3] http://man7.org/linux/man-pages/man3/pthread_cancel.3.html [4] https://docs.microsoft.com/en-us/windows/desktop/api/processthreadsapi/nf-pr...
IME in many of these cases you're better off using asyncio instead. -- Ryan https://refi64.com/ On Jun 20, 2019, 12:14 AM -0500, Matúš Valo <matusvalo@gmail.com>, wrote:
On Jun 19, 2019, at 22:14, Matúš Valo <matusvalo@gmail.com> wrote:
Hi All,
Currently it is not possible to "kill" thread which is blocked. The rationale for this is handling situations when thread is blocked - e.g. when thread is quering DB when lock occurred on Database.
pthread library and Windows API contains mechanisms for forced termination of threads - see [3] and [4].
The Windows function starts off its description with “TerminateThread is a dangerous function that should only be used in the most extreme cases.” The POSIX function may or may not be safer. By default it is—but only because by default it doesn’t actually terminate the thread, it just sends a signal which sets a flag to be checked at the next cancel point. If the blocked operation can’t be interrupted by the signal, it stays blocked. If it can be interrupted, then you could have just signaled it instead, which is safer and easier to manage, especially from Python. Either way, there’s no way to safely clean up after it. (Deleting the threading library’s own reference to its Thread object doesn’t solve that.) And if your program regularly needs to terminate threads, your program almost certainly needs to be redesigned. What you almost always want is signals, asyncio (a lot more operations can be canceled, and it’s usually less bad when then can’t), or something specific to your operation (e.g., a database library might have a connection-closing method, or maybe you can just close a file handle). The advantage to these solutions is that they generally mean the blocking operation exits immediately by raising an exception, which the thread can handle Pythonically to make sure everything ends up in the right state (usually just letting the exception propagate up to the top-level thread function and exit normally is all you need). When you really do need to occasionally terminate something blocking, put it in a process rather than a thread. Processes can be killed, the OS automatically cleans up everything except persistent objects, and there’s no state being shared with your main process that could be corrupted. You can still have problems (e.g., if you’re using lockfiles manually, and you hard-kill a process that’s created the lockfile, it’s left behind), but with processes things are safe except in special conditions, as opposed to threads where things are broken except in special conditions.
Thank you for your reply. The use case is simple just to offload SQL query to separate thread to not block main thread. The problem is handling situation when thread is blocked due I/O to Database. I am not aware about solution like timeout of query etc. but maybe I have missed something.
On Thu, 20 Jun 2019 at 06:15, Matúš Valo <matusvalo@gmail.com> wrote:
Back in the day we used the .NET API for king threads from IronPython. It made threads so much saner to work with. It's been an egregious lack in the Python thread API for years and every time it's brought up people rush to say you shouldn't do it as what you really want to do is something else. Michael
-- Michael Foord Python Consultant, Contractor and Trainer https://agileabstractions.com/
I have found out at least this link: http://www.voidspace.org.uk/ironpython/threading.shtml There is also example of aborting thread. Matus
On Thu, 20 Jun 2019 at 16:11, Matúš Valo <matusvalo@gmail.com> wrote:
Thanks, good reference 😀 It also mentions the same API in Java. Michael
-- Michael Foord Python Consultant, Contractor and Trainer https://agileabstractions.com/
On 20/06/2019 16:10, Matúš Valo wrote:
""" The next sections provide some simple examples of threading with IronPython, and illustrates using Thread.Abort. This doesn't mean that signalling to threads when to stop isn't *generally* a better technique, but (especially with a shared nothing model) it can sometimes be safe to abort threads. """ There are almost always consequences to killing threads. Anyone who thinks its an easy fix for pretty much anything is likely to suffer from subtle and mysterious side-effects :-( -- Rhodri James *-* Kynesim Ltd
On Thu, 20 Jun 2019 at 13:03, Robert Collins <robertc@robertcollins.net> wrote:
Do you have a link to that? Googling king .net threads got me some strange results 😁.
Auto-correct. Killing. Thread.abort I believe. I don't think it's very hard to find.
What made it particularly sane?
A straightforward API that works without fuss or excuses. It works by raising an exception in the target thread, which the thread is free to handle (usually for cleanup and then reraise). Michael
Michael Foord Python Consultant, Contractor and Trainer https://agileabstractions.com/
On Thu, Jun 20, 2019 at 8:21 AM Michael Foord <fuzzyman@gmail.com> wrote:
It works by raising an exception in the target thread, which the thread is free to handle (usually for cleanup and then reraise).
Sure, those are the right semantics. How does it stop blocking I/O though? Suppose the thread is waiting for a server to return a response which just isn't ever going to come, but the connection somehow is kept open by the other side? -- --Guido van Rossum (python.org/~guido) Pronouns: he/him/his (why is my pronoun here?)
On Thu, 20 Jun 2019 at 16:33, Guido van Rossum <guido@python.org> wrote:
Sorry, resending to list as well. It used to be on the CLR getting back control. So it couldn't handle that case. (.NET 1.1). https://jonskeet.uk/csharp/threads/abort.html It has since been improved. It still blocks on the execution of unmanaged code (or computation in a finally block handling the ThreadAbortException), but blocking IO can be interrupted: https://docs.microsoft.com/en-us/dotnet/api/system.threading.thread.abort?vi... If Abort is called on a thread that is blocked or is sleeping, the thread is interrupted and then aborted. This SO question on the topic says: https://stackoverflow.com/questions/365370/proper-way-to-stop-tcplistener Thread.Abort() There are 2 things I see you could do. 1 is that if you have started this TcpListener thread from another you can simply call Thread.Abort instance method on the thread which will cause a threadabortexception to be thrown within the blocking call and walk up the stack. Michael
-- Michael Foord Python Consultant, Contractor and Trainer https://agileabstractions.com/
Generally, threads don't have a notion of non-cooperative thread termination. This is because (unlike processes) threads share address spaces, and an unexpected termination of a thread can leave memory in arbitrary and unexpected states. (For example, what if one thread was holding a mutex when it got killed?) The POSIX threading model *does* include the ability to send a signal to a particular thread using pthread_kill(). What that does is cause the process' globally-registered signal handler for that signal to be invoked in that particular thread. With the default handlers, though, these tend to have process-wide effects; e.g., if you pthread_kill(tid, 15) with the default handler, it will simply kill the process. This doesn't make as much sense in the context of Python's signal handling (which is a layer on top of POSIX signal handling) so signals.pthread_kill() isn't a very useful function. In practice (and I actually had to deal with exactly this problem in my own code just last week!) there are two approaches that tend to work: (1) Cooperative scheduling: Your code has an explicit place within the threads where they check for abort signals and handle them appropriately. For example, if your threads are each running event listeners, you might post a "cancel" event on the main bus; or alternately, you could use a threading.Event to signal everyone to shut down. (2) Separate address spaces: If (1) isn't possible for some reason, and you need to literally be able to kill off a process noncooperatively -- say, the purpose of your code is to be an intermediate "harness" library which runs someone else's long-running functions in a thread, and it's not possible to require all of that code to obey a cooperative protocol -- then use processes instead of threads. In many cases, you can achieve this with built-in Python libraries like multiprocessing and ProcessPoolExecutor, and in this case, you can terminate processes simply by calling Popen.send_signal() or the like. In a few cases (e.g., if you're using gRPC, whose client library is incompatible with those for complicated reasons) you'll have to fork() and exec() yourself, typically by using the subprocess.Popen library directly. Yonatan On Thu, Jun 20, 2019 at 10:16 AM Michael Foord <fuzzyman@gmail.com> wrote:
On Thu, 27 Jun 2019 at 20:53, Yonatan Zunger <zunger@humu.com> wrote:
That's precisely why thread cancellation in managed languages (like Python is) raise an exception to terminate the the thread and honour finally blocks.
This is not what is being suggested. Read about the semantics of thread killing in C# and Java.
In practice (and I actually had to deal with exactly this problem in my own code just last week!) there are two approaches that tend to work:
Yet again someone who doesn't understand what is being proposed saying it isn't needed. Michael
Michael Foord Python Consultant, Contractor and Trainer https://agileabstractions.com/
On Thu, Jun 27, 2019 at 1:36 PM Michael Foord <fuzzyman@gmail.com> wrote:
There's one issue that Yonatan's post reminded me of though. Currently a thread that's locked in a mutex.acquire() operation cannot be interrupted by a signal at the Python level (unlike I/O, which can be interrupted -- IIRC Victor spent a lot of time making this work). There's a workaround (specify a timeout) but this is still something that would have to be solved for this to be useful. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him/his **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
Although thinking about it more, and in the context of this thread, the idea of an operation to "raise exception X in thread Y" doesn't seem unreasonable. After all, this is basically what happens to the main thread from the signal handler already; the C signal handler caches a bit that gets picked up by PyErr_CheckSignals(), which raises things like KeyboardInterrupt at completely arbitrary points w.r.t. user code. One possible approach (at a very schematic level, not thinking about impl details yet) would be to have a function like Thread.raise(e: BaseException) which stuffs the exception into thread-local storage, which then gets checked at similar times to signals. (Probably not the exact same times since I'm sure there's plenty of code depending implicitly on that function being a noop outside the main thread) Basically, it would be having the Python interpreter doing the cooperative part of cross-thread interruption, so that it would look non-cooperative from the user side. You could even have a Thread.raiseWhen to do the equivalent of setting an alarm. This would definitely be a "use it only if you really mean it" function, but it could be quite useful. Implementing things like watchdog timers is hard right now because of a lack of this sort of feature, but this could open up some nice options. On Thu, Jun 27, 2019 at 2:17 PM Guido van Rossum <guido@python.org> wrote:
On Thursday, June 27, 2019, 03:40:56 PM PDT, Yonatan Zunger <zunger@humu.com> wrote:
One possible approach (at a very schematic level, not thinking about impl details yet) would be to have a function like Thread.raise(e: BaseException) which stuffs the exception into thread-local storage, which then gets checked at similar times to signals. (Probably not the exact same times since I'm sure there's plenty of code depending implicitly on that function being a noop outside the main thread) Basically, it would be having the Python interpreter doing the cooperative part of cross-thread interruption, so that it would look non-cooperative from the user side. You could even have a Thread.raiseWhen to do the equivalent of setting an alarm.
You could build a hacky version of this today in just a few lines, something like this: class AbortableThread(threading.Thread): def __init__(self, *args, **kw): super().__init__(*args, **kw) self._e = None def _trace(self, frame, event, arg): if self._e: raise self._e() return self._trace def run(self, *args, **kw): sys.settrace(self._trace) super().run(*args, **kw) def throw(self, e): self._e = e You wouldn't want to do this in practice, but for playing with the API to see if it feels right/usable, it should be fine.
On Jun 27, 2019, at 13:36, Michael Foord <fuzzyman@gmail.com> wrote:
On Thu, 27 Jun 2019 at 20:53, Yonatan Zunger <zunger@humu.com> wrote: Generally, threads don't have a notion of non-cooperative thread termination.
That's precisely why thread cancellation in managed languages (like Python is) raise an exception to terminate the the thread and honour finally blocks.
And that’s precisely why your proposal to use TerminateThread on Windows can’t work. And why .NET Thread.Abort, etc., don’t use TerminateThread—and, in fact, don’t break out of most blocking calls.
The POSIX threading model does include the ability to send a signal to a particular thread using pthread_kill().
This is not what is being suggested. Read about the semantics of thread killing in C# and Java.
While Yonatan is talking about the old pthread_kill function rather than the modern one you suggested, there are still similar issues. It still works by sending a signal under the covers, it just provides a much nicer API (but one that nobody uses—and in particular, Python doesn’t) to handle that signal indirectly, by managing thread cancellation state and type and cleanup functions.
IME in many of these cases you're better off using asyncio instead. -- Ryan https://refi64.com/ On Jun 20, 2019, 12:14 AM -0500, Matúš Valo <matusvalo@gmail.com>, wrote:
On Jun 19, 2019, at 22:14, Matúš Valo <matusvalo@gmail.com> wrote:
Hi All,
Currently it is not possible to "kill" thread which is blocked. The rationale for this is handling situations when thread is blocked - e.g. when thread is quering DB when lock occurred on Database.
pthread library and Windows API contains mechanisms for forced termination of threads - see [3] and [4].
The Windows function starts off its description with “TerminateThread is a dangerous function that should only be used in the most extreme cases.” The POSIX function may or may not be safer. By default it is—but only because by default it doesn’t actually terminate the thread, it just sends a signal which sets a flag to be checked at the next cancel point. If the blocked operation can’t be interrupted by the signal, it stays blocked. If it can be interrupted, then you could have just signaled it instead, which is safer and easier to manage, especially from Python. Either way, there’s no way to safely clean up after it. (Deleting the threading library’s own reference to its Thread object doesn’t solve that.) And if your program regularly needs to terminate threads, your program almost certainly needs to be redesigned. What you almost always want is signals, asyncio (a lot more operations can be canceled, and it’s usually less bad when then can’t), or something specific to your operation (e.g., a database library might have a connection-closing method, or maybe you can just close a file handle). The advantage to these solutions is that they generally mean the blocking operation exits immediately by raising an exception, which the thread can handle Pythonically to make sure everything ends up in the right state (usually just letting the exception propagate up to the top-level thread function and exit normally is all you need). When you really do need to occasionally terminate something blocking, put it in a process rather than a thread. Processes can be killed, the OS automatically cleans up everything except persistent objects, and there’s no state being shared with your main process that could be corrupted. You can still have problems (e.g., if you’re using lockfiles manually, and you hard-kill a process that’s created the lockfile, it’s left behind), but with processes things are safe except in special conditions, as opposed to threads where things are broken except in special conditions.
Thank you for your reply. The use case is simple just to offload SQL query to separate thread to not block main thread. The problem is handling situation when thread is blocked due I/O to Database. I am not aware about solution like timeout of query etc. but maybe I have missed something.
On Thu, 20 Jun 2019 at 06:15, Matúš Valo <matusvalo@gmail.com> wrote:
Back in the day we used the .NET API for king threads from IronPython. It made threads so much saner to work with. It's been an egregious lack in the Python thread API for years and every time it's brought up people rush to say you shouldn't do it as what you really want to do is something else. Michael
-- Michael Foord Python Consultant, Contractor and Trainer https://agileabstractions.com/
I have found out at least this link: http://www.voidspace.org.uk/ironpython/threading.shtml There is also example of aborting thread. Matus
On Thu, 20 Jun 2019 at 16:11, Matúš Valo <matusvalo@gmail.com> wrote:
Thanks, good reference 😀 It also mentions the same API in Java. Michael
-- Michael Foord Python Consultant, Contractor and Trainer https://agileabstractions.com/
On 20/06/2019 16:10, Matúš Valo wrote:
""" The next sections provide some simple examples of threading with IronPython, and illustrates using Thread.Abort. This doesn't mean that signalling to threads when to stop isn't *generally* a better technique, but (especially with a shared nothing model) it can sometimes be safe to abort threads. """ There are almost always consequences to killing threads. Anyone who thinks its an easy fix for pretty much anything is likely to suffer from subtle and mysterious side-effects :-( -- Rhodri James *-* Kynesim Ltd
On Thu, 20 Jun 2019 at 13:03, Robert Collins <robertc@robertcollins.net> wrote:
Do you have a link to that? Googling king .net threads got me some strange results 😁.
Auto-correct. Killing. Thread.abort I believe. I don't think it's very hard to find.
What made it particularly sane?
A straightforward API that works without fuss or excuses. It works by raising an exception in the target thread, which the thread is free to handle (usually for cleanup and then reraise). Michael
Michael Foord Python Consultant, Contractor and Trainer https://agileabstractions.com/
On Thu, Jun 20, 2019 at 8:21 AM Michael Foord <fuzzyman@gmail.com> wrote:
It works by raising an exception in the target thread, which the thread is free to handle (usually for cleanup and then reraise).
Sure, those are the right semantics. How does it stop blocking I/O though? Suppose the thread is waiting for a server to return a response which just isn't ever going to come, but the connection somehow is kept open by the other side? -- --Guido van Rossum (python.org/~guido) Pronouns: he/him/his (why is my pronoun here?)
On Thu, 20 Jun 2019 at 16:33, Guido van Rossum <guido@python.org> wrote:
Sorry, resending to list as well. It used to be on the CLR getting back control. So it couldn't handle that case. (.NET 1.1). https://jonskeet.uk/csharp/threads/abort.html It has since been improved. It still blocks on the execution of unmanaged code (or computation in a finally block handling the ThreadAbortException), but blocking IO can be interrupted: https://docs.microsoft.com/en-us/dotnet/api/system.threading.thread.abort?vi... If Abort is called on a thread that is blocked or is sleeping, the thread is interrupted and then aborted. This SO question on the topic says: https://stackoverflow.com/questions/365370/proper-way-to-stop-tcplistener Thread.Abort() There are 2 things I see you could do. 1 is that if you have started this TcpListener thread from another you can simply call Thread.Abort instance method on the thread which will cause a threadabortexception to be thrown within the blocking call and walk up the stack. Michael
-- Michael Foord Python Consultant, Contractor and Trainer https://agileabstractions.com/
Generally, threads don't have a notion of non-cooperative thread termination. This is because (unlike processes) threads share address spaces, and an unexpected termination of a thread can leave memory in arbitrary and unexpected states. (For example, what if one thread was holding a mutex when it got killed?) The POSIX threading model *does* include the ability to send a signal to a particular thread using pthread_kill(). What that does is cause the process' globally-registered signal handler for that signal to be invoked in that particular thread. With the default handlers, though, these tend to have process-wide effects; e.g., if you pthread_kill(tid, 15) with the default handler, it will simply kill the process. This doesn't make as much sense in the context of Python's signal handling (which is a layer on top of POSIX signal handling) so signals.pthread_kill() isn't a very useful function. In practice (and I actually had to deal with exactly this problem in my own code just last week!) there are two approaches that tend to work: (1) Cooperative scheduling: Your code has an explicit place within the threads where they check for abort signals and handle them appropriately. For example, if your threads are each running event listeners, you might post a "cancel" event on the main bus; or alternately, you could use a threading.Event to signal everyone to shut down. (2) Separate address spaces: If (1) isn't possible for some reason, and you need to literally be able to kill off a process noncooperatively -- say, the purpose of your code is to be an intermediate "harness" library which runs someone else's long-running functions in a thread, and it's not possible to require all of that code to obey a cooperative protocol -- then use processes instead of threads. In many cases, you can achieve this with built-in Python libraries like multiprocessing and ProcessPoolExecutor, and in this case, you can terminate processes simply by calling Popen.send_signal() or the like. In a few cases (e.g., if you're using gRPC, whose client library is incompatible with those for complicated reasons) you'll have to fork() and exec() yourself, typically by using the subprocess.Popen library directly. Yonatan On Thu, Jun 20, 2019 at 10:16 AM Michael Foord <fuzzyman@gmail.com> wrote:
On Thu, 27 Jun 2019 at 20:53, Yonatan Zunger <zunger@humu.com> wrote:
That's precisely why thread cancellation in managed languages (like Python is) raise an exception to terminate the the thread and honour finally blocks.
This is not what is being suggested. Read about the semantics of thread killing in C# and Java.
In practice (and I actually had to deal with exactly this problem in my own code just last week!) there are two approaches that tend to work:
Yet again someone who doesn't understand what is being proposed saying it isn't needed. Michael
Michael Foord Python Consultant, Contractor and Trainer https://agileabstractions.com/
On Thu, Jun 27, 2019 at 1:36 PM Michael Foord <fuzzyman@gmail.com> wrote:
There's one issue that Yonatan's post reminded me of though. Currently a thread that's locked in a mutex.acquire() operation cannot be interrupted by a signal at the Python level (unlike I/O, which can be interrupted -- IIRC Victor spent a lot of time making this work). There's a workaround (specify a timeout) but this is still something that would have to be solved for this to be useful. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him/his **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
Although thinking about it more, and in the context of this thread, the idea of an operation to "raise exception X in thread Y" doesn't seem unreasonable. After all, this is basically what happens to the main thread from the signal handler already; the C signal handler caches a bit that gets picked up by PyErr_CheckSignals(), which raises things like KeyboardInterrupt at completely arbitrary points w.r.t. user code. One possible approach (at a very schematic level, not thinking about impl details yet) would be to have a function like Thread.raise(e: BaseException) which stuffs the exception into thread-local storage, which then gets checked at similar times to signals. (Probably not the exact same times since I'm sure there's plenty of code depending implicitly on that function being a noop outside the main thread) Basically, it would be having the Python interpreter doing the cooperative part of cross-thread interruption, so that it would look non-cooperative from the user side. You could even have a Thread.raiseWhen to do the equivalent of setting an alarm. This would definitely be a "use it only if you really mean it" function, but it could be quite useful. Implementing things like watchdog timers is hard right now because of a lack of this sort of feature, but this could open up some nice options. On Thu, Jun 27, 2019 at 2:17 PM Guido van Rossum <guido@python.org> wrote:
On Thursday, June 27, 2019, 03:40:56 PM PDT, Yonatan Zunger <zunger@humu.com> wrote:
One possible approach (at a very schematic level, not thinking about impl details yet) would be to have a function like Thread.raise(e: BaseException) which stuffs the exception into thread-local storage, which then gets checked at similar times to signals. (Probably not the exact same times since I'm sure there's plenty of code depending implicitly on that function being a noop outside the main thread) Basically, it would be having the Python interpreter doing the cooperative part of cross-thread interruption, so that it would look non-cooperative from the user side. You could even have a Thread.raiseWhen to do the equivalent of setting an alarm.
You could build a hacky version of this today in just a few lines, something like this: class AbortableThread(threading.Thread): def __init__(self, *args, **kw): super().__init__(*args, **kw) self._e = None def _trace(self, frame, event, arg): if self._e: raise self._e() return self._trace def run(self, *args, **kw): sys.settrace(self._trace) super().run(*args, **kw) def throw(self, e): self._e = e You wouldn't want to do this in practice, but for playing with the API to see if it feels right/usable, it should be fine.
On Jun 27, 2019, at 13:36, Michael Foord <fuzzyman@gmail.com> wrote:
On Thu, 27 Jun 2019 at 20:53, Yonatan Zunger <zunger@humu.com> wrote: Generally, threads don't have a notion of non-cooperative thread termination.
That's precisely why thread cancellation in managed languages (like Python is) raise an exception to terminate the the thread and honour finally blocks.
And that’s precisely why your proposal to use TerminateThread on Windows can’t work. And why .NET Thread.Abort, etc., don’t use TerminateThread—and, in fact, don’t break out of most blocking calls.
The POSIX threading model does include the ability to send a signal to a particular thread using pthread_kill().
This is not what is being suggested. Read about the semantics of thread killing in C# and Java.
While Yonatan is talking about the old pthread_kill function rather than the modern one you suggested, there are still similar issues. It still works by sending a signal under the covers, it just provides a much nicer API (but one that nobody uses—and in particular, Python doesn’t) to handle that signal indirectly, by managing thread cancellation state and type and cleanup functions.
participants (9)
-
Andrew Barnert -
fuzzyman＠gmail.com -
Guido van Rossum -
Matúš Valo -
Michael Foord
-
Rhodri James -
Robert Collins -
Ryan Gonzalez -
Yonatan Zunger