[Python-Dev] Status of thread cancellation
glyph at divmod.com
glyph at divmod.com
Thu Mar 15 19:25:23 CET 2007
On 04:24 pm, martin at v.loewis.de wrote:
>Jean-Paul Calderone schrieb:
>>>I inferred from Martin's proposal that he
>>>expected the thread to be able to catch the exception. Perhaps he
>>>elaborate on what cleanup actions the dying thread will be allowed to
>>Perhaps he can. Hopefully, he can specifically address these points:
>> 1. A thread can throw a ThreadDeath exception almost anywhere. All
>> synchronized methods and blocks would have to be studied in
>> detail, with this in mind.
>> 2. A thread can throw a second ThreadDeath exception while cleaning
>> from the first (in the catch or finally clause). Cleanup would
>> to repeated till it succeeded. The code to ensure this would be
>Clearly, a thread need to have its finally blocks performed in response
>to a cancellation request. These issues are real, however, they apply
>to any asynchronous exception, not just to thread cancellation.
To be sure, the problem does apply to all asynchronous exceptions.
That's why it is generally understood that a program which has received
an asynchronous exception cannot continue.
>In Python, we already have an asynchronous exception:
>This suffers from the same problems: a KeyboadInterrupt also can occur
>at any point, interrupting code in the middle of its finally-blocks.
>The other exception that is nearly-asynchronous is OutOfMemoryError,
>which can occur at nearly any point (but of course, never occurs in
KeyboardInterrupt and MemoryError share a common feature which forced
thread termination does not: nobody reasonably expects the program to
keep running after they have been raised. Indeed, programs are written
with the expectation that MemoryError will never occur, and if it does,
the user is not surprised to find them in an inconsistent state. In any
situation where a MemoryError may reasonably be expected - that is to
say, a specific, large allocation of a single block of memory - it can
be trapped as if it were not asynchronous.
Long-running Python programs which expect to need to do serious clean-up
in the face of interrupts, in fact, block KeyboardInterrupt by
registering their own interrupt handlers (Zope, Twisted).
Developers who think they want thread cancellation inevitably believe
they can, if they are "sufficiently careful", implement operating-
system-like scheduling features by starting arbitrary user code and then
providing "terminate", "pause", and "resume" commands. That was the
original intent of these (now removed) Java APIs, and that is why they
were removed: you can't do this. It's impossible.
Asynchronous exceptions are better than immediate termination because
they allow for code which is allocating scarce or fragile resources to
have a probabilistically better chance of cleaning up. However, nobody
writes code like this:
def addSomeStuff(self, volume, mass):
self.volume += volume
self.mass += mass
self.volume -= volume
and nobody is going to start if the language provides thread
termination. Async-Exception-Safe Python code is, and will be, as rare
as POSIX Async-Safe functions, which means at best you will be able to
call a thread cancellation API and have it _appear_ to work in some
circumstances. In any system which uses Python code not explicitly
designed to support asynchronous exceptions (let's say, the standard
library) it will be completely impossible to write correct code.
I'm not a big fan of shared-state-threading, but it does allow for a
particular programming model. Threading provides you some guarantees.
You can't poke around on the heap, but you know that your stack, and
your program counter, are inviolate. You can reason about, if not quite
test, the impact of sharing a piece of state on the heap; its
destructive methods need to be synchronized along with the read methods
that interact with it.
Asynchronous exceptions destroy all hope of sanity. Your program might
suddenly perform a nonlocal exit _anywhere_ except, maybe, inside a
"finally" block. This literally encourages some people that program in
environments where asynchronous exceptions are possible (.NET, in
particular) to put huge chunks of application code inside finally
blocks. They generally look like this:
// entire application here
because that is really the only way you can hope to write code that will
function robustly in such an environment.
>So yes, it would be good if Python's exception handling supported
>asynchronous exceptions in a sensible way. I have to research somewhat
>more, but I think the standard solution to the problem in operating
>system (i.e. disabling interrupts at certain points, explicitly
>due to code or implicitly as a result of entering the interrupt
>handler) may apply.
For one thing, the Python core code is not operating system kernel code
and it is unlikely that the techniques found there will apply.
Interrupts, in particular, are nothing at all like exceptions, and have
a completely different impact on running code (which, since it is kernel
code, is written much more carefully and under an entirely different set
of constraints than Python application, or even framework, code).
Can you suggest any use-cases for thread termination which will *not*
result in a completely broken and unpredictable heap after the thread
has died? If you can think of such a case, are you sure it wouldn't be
better served by a set of threads communicating over queues and sending
'Stop' objects to each other to indicate termination at a point in queue
rather than a forced termination via exception?
Just in case it's not clear from the other things I've said: this is a
terrible, terrible idea, and I am shocked that it is even being
*considered* for inclusion in Python. As a foolish youth, I wasted many
months trying to get a program that used Java's (then not deprecated)
asynchronous exception APIs to behave properly. It wasn't possible
then, and it isn't possible now.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Python-Dev