Thread exceptions and interruption
One of the core problems with threading is what to do with exceptions and how to gracefully exit when one goes unhandled. My approach is to replace the independently spawned threads with "branches" off of your main thread's call stack. The standard example looks like this[1]: def handle_client(conn, addr): with conn: ... def accept_loop(server_conn): with branch() as clients: with server_conn: while True: clients.add(handle_client, *server_conn.accept()) The call stack will look something like this: main - accept_loop - server_conn.accept |- handle_client \- handle_client Here I use a with-statement[2] to create a branch point. The branch point collects any exceptions from its children and interrupts the children when the first exception occurs. Interruption is done somewhat similarly to posix cancellation; participating functions react to it. However, I raise an Interrupted exception, which can lead to much more graceful cleanup than posix cancellation. ;) The __exit__ portion of branch's with-statement blocks until all child threads have exited. It then reraises the exception, if any, or wraps it in MultipleError if several occurred. The branch construct serves only simple needs. It does not attempt to limit the number of threads to the number of cores available, nor any related tricks. Those can be added as a separate tool (perhaps wrapping branch.) Thoughts? Competing ideas? Disagreement that it's a "core problem" at all? ;) [1] I've previously (in private mostly) referred to the branch() function as collate(). I've recently decided to rename it. [2] Unfortunately, a with-statement lacks all the invariants that would be desirable for the branch construct. It also has no direct way of handling generators-as-context-managers that themselves use branches. -- Adam Olsen, aka Rhamphoryncus
Adam Olsen wrote:
It sounds like you're proposing that a thread can be interrupted at any time. The Java developers realised long ago that this is completely unworkable and deprecated their implementation: http://java.sun.com/j2se/1.3/docs/guide/misc/threadPrimitiveDeprecation.html Please disregard if I misunderstood your approach :-) Alex.
Regarding the issue of exceptions in threads, I indeed see it as a non-issue. It's easy enough to develop a subclass of threading.Thread which catches any exceptions raised by run(), and stores the exception as an instance variable from which it can be retrieved after join() succeeds. Regarding the proposal of branching the call stack, it reminds me too much of the problems one has when a fork()'ed child raises an exception which ends up being handled by an exception handler higher up in the parent's call stack (which has been faithfully copied into the child process by fork()). That has proven a major problem, leading to various warnings to always catch all exceptions and call os._exit() upon problems. I realize you're not proposing exactly that. I also admit I don't exactly understand how you plan to deal with the situation where one thread raises an exception which the spawning location fails to handle, while another thread is still running (but may raise another exception later). Is the spawning thread unwound? Then what's left to catch the second thread's exception? But all in all it gives me the heebie-jeebies. Finally, may I suggest that you're perhaps too much in love with the with-statement? --Guido On 9/18/07, Adam Olsen <rhamph@gmail.com> wrote:
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
On 9/19/07, Guido van Rossum <guido@python.org> wrote:
I don't see what you're getting at. No stack copying is done so fork is irrelevant and the spawning threads *always* blocks until all of its child threads have exited. Let's try a simpler example, without the main thread (which isn't special for exception purposes): (Make sure to look at it with a monospace font) / baz foo - bar +- baz \ baz bar encapsulates several threads. It makes no sense unravel the call tree while a lower portion of it still exists, so it must wait. If there is a failure, bar will politely tell all 3 baz functions to exit, but they probably won't listen (unless they're calling I/O). If necessary bar will wait forever. foo never sees any of this. It is completely hidden within bar.
Finally, may I suggest that you're perhaps too much in love with the with-statement?
I've a preference for writing as a library, rather than with new syntax. ;) -- Adam Olsen, aka Rhamphoryncus
On 9/19/07, Adam Olsen <rhamph@gmail.com> wrote:
So what happens if the first baz thread raises an exception that bar isn't handling? I suppose it first waits until all baz threads are done, but then the question is still open. Does it percolate up to foo? What if two or more baz threads raise exceptions? How does foo see these? -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On 9/19/07, Guido van Rossum <guido@python.org> wrote:
bar itself doesn't see it until *after* they've all exited. The branch construct holds it until all child threads have exited. There is no way to get weird stack unwinding. If multiple exceptions occur they get encapsulated in a MultipleError exception. -- Adam Olsen, aka Rhamphoryncus
On 9/19/07, Guido van Rossum <guido@python.org> wrote:
Perhaps a better question then: do you think it correctly handling errors is a significant part of what makes threads hard today? My focus has always been on making "simultaneous activities" easier to manage. Removing the GIL is just a free bonus from making independent tasks really be independent. -- Adam Olsen, aka Rhamphoryncus
On 9/19/07, Adam Olsen <rhamph@gmail.com> wrote:
If you're talking about unhandled exceptions, no, that's absolutely a non-issue. The real issues are race conditions, deadlocks, livelocks etc. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On 9/19/07, Guido van Rossum <guido@python.org> wrote:
I guess the bottom line here is that, since none of the proposed solutions magically eliminate race conditions, deadlocks, livelocks, etc, we'll need to try them in the field for quite some time before it's clear if the ways they do make things better have any significant effects in reducing the core problems. In other words, I (and the other pundits) should implement our ideas in a forked python, and not propose merging back until we've got a large user base with a proven track record. Even if that's not as much fun. ;) -- Adam Olsen, aka Rhamphoryncus
On 9/19/07, Adam Olsen <rhamph@gmail.com> wrote:
Agreed. Though race conditions become less of a problem if you don't have fine-grained memory sharing (where you always hope you can get away without a lock -- just Google for "double-checked locking" :-). And deadlocks can be fought quite effectively by a runtime layer that detects them, plus strategies for forcing lock acquisition order. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Adam Olsen wrote:
It sounds like you're proposing that a thread can be interrupted at any time. The Java developers realised long ago that this is completely unworkable and deprecated their implementation: http://java.sun.com/j2se/1.3/docs/guide/misc/threadPrimitiveDeprecation.html Please disregard if I misunderstood your approach :-) Alex.
Regarding the issue of exceptions in threads, I indeed see it as a non-issue. It's easy enough to develop a subclass of threading.Thread which catches any exceptions raised by run(), and stores the exception as an instance variable from which it can be retrieved after join() succeeds. Regarding the proposal of branching the call stack, it reminds me too much of the problems one has when a fork()'ed child raises an exception which ends up being handled by an exception handler higher up in the parent's call stack (which has been faithfully copied into the child process by fork()). That has proven a major problem, leading to various warnings to always catch all exceptions and call os._exit() upon problems. I realize you're not proposing exactly that. I also admit I don't exactly understand how you plan to deal with the situation where one thread raises an exception which the spawning location fails to handle, while another thread is still running (but may raise another exception later). Is the spawning thread unwound? Then what's left to catch the second thread's exception? But all in all it gives me the heebie-jeebies. Finally, may I suggest that you're perhaps too much in love with the with-statement? --Guido On 9/18/07, Adam Olsen <rhamph@gmail.com> wrote:
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
On 9/19/07, Guido van Rossum <guido@python.org> wrote:
I don't see what you're getting at. No stack copying is done so fork is irrelevant and the spawning threads *always* blocks until all of its child threads have exited. Let's try a simpler example, without the main thread (which isn't special for exception purposes): (Make sure to look at it with a monospace font) / baz foo - bar +- baz \ baz bar encapsulates several threads. It makes no sense unravel the call tree while a lower portion of it still exists, so it must wait. If there is a failure, bar will politely tell all 3 baz functions to exit, but they probably won't listen (unless they're calling I/O). If necessary bar will wait forever. foo never sees any of this. It is completely hidden within bar.
Finally, may I suggest that you're perhaps too much in love with the with-statement?
I've a preference for writing as a library, rather than with new syntax. ;) -- Adam Olsen, aka Rhamphoryncus
On 9/19/07, Adam Olsen <rhamph@gmail.com> wrote:
So what happens if the first baz thread raises an exception that bar isn't handling? I suppose it first waits until all baz threads are done, but then the question is still open. Does it percolate up to foo? What if two or more baz threads raise exceptions? How does foo see these? -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On 9/19/07, Guido van Rossum <guido@python.org> wrote:
bar itself doesn't see it until *after* they've all exited. The branch construct holds it until all child threads have exited. There is no way to get weird stack unwinding. If multiple exceptions occur they get encapsulated in a MultipleError exception. -- Adam Olsen, aka Rhamphoryncus
On 9/19/07, Guido van Rossum <guido@python.org> wrote:
Perhaps a better question then: do you think it correctly handling errors is a significant part of what makes threads hard today? My focus has always been on making "simultaneous activities" easier to manage. Removing the GIL is just a free bonus from making independent tasks really be independent. -- Adam Olsen, aka Rhamphoryncus
On 9/19/07, Adam Olsen <rhamph@gmail.com> wrote:
If you're talking about unhandled exceptions, no, that's absolutely a non-issue. The real issues are race conditions, deadlocks, livelocks etc. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On 9/19/07, Guido van Rossum <guido@python.org> wrote:
I guess the bottom line here is that, since none of the proposed solutions magically eliminate race conditions, deadlocks, livelocks, etc, we'll need to try them in the field for quite some time before it's clear if the ways they do make things better have any significant effects in reducing the core problems. In other words, I (and the other pundits) should implement our ideas in a forked python, and not propose merging back until we've got a large user base with a proven track record. Even if that's not as much fun. ;) -- Adam Olsen, aka Rhamphoryncus
On 9/19/07, Adam Olsen <rhamph@gmail.com> wrote:
Agreed. Though race conditions become less of a problem if you don't have fine-grained memory sharing (where you always hope you can get away without a lock -- just Google for "double-checked locking" :-). And deadlocks can be fought quite effectively by a runtime layer that detects them, plus strategies for forcing lock acquisition order. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
participants (3)
-
Adam Olsen -
Alex Holkner -
Guido van Rossum