I haven't looked, but is it possible to make it raise an arbitrary exception? Last time I read the PyErr_CheckSignals code it looked possible. (That was yesterday, actually, regarding problems with the SSL module and KeyboardInterrupts).

The interrupt module currently uses PyErr_SetInterrupt(), which can only raise KeyboardInterrupt. Let's not generalize this until there's a need. Then, let whoever has the need add the generalization. I'm actually a bit wary of such a generalization, because KeyboardInterrupt is currently the only *asynchronous* exception that code needs to worry about. There is plenty of code that gives KeyboardInterrupt special treatment, typically of the form: while MoreWorkToDo: try: ...run a chunk of work which may raise arbitrary exceptions... except KeyboardInterrupt: raise # Don't keep going in this case except: ...report the error and continue... --Guido van Rossum (home page: http://www.python.org/~guido/)

...

I'm actually a bit wary of such a generalization, because KeyboardInterrupt is currently the only *asynchronous* exception that code needs to worry about. There is plenty of code that gives KeyboardInterrupt special treatment, typically of the form:

while MoreWorkToDo: try: ...run a chunk of work which may raise arbitrary exceptions... except KeyboardInterrupt: raise # Don't keep going in this case except: ...report the error and continue...

What about SystemExit?

It was only an example! What should be done with SystemExit depends on what the application is trying to do. In an interactive Python shell, you probably want to re-raise it. But in other cases, SystemExit is an error like all others. (For example, consider a loop that walks a directory tree and runs testcases found in files named "test.py".) --Guido van Rossum (home page: http://www.python.org/~guido/)

Not to be pedantic, but isn't the range of interpretations of SystemExit exactly analogous to that of KeyboardInterrupt?

I don't understand what you mean here. I thought I gave an example where it wasn't.

Also remember that pounding Control-C isn't the only way for a process or thread to get a SIGINT (and thus a KeyboardInterrupt).

I'm aware of kill, yes. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)

In that past, there have been a few times where it would have been very convenient to terminate/signal another thread by posting an exception to it. Using KeyboardInterrupt to do so is okay, although it seems like something of a kludge.

This gets into the general discussion about whether threads should be stoppable. Also, this mechanism can *only* interrupt the main thread. (That's even true of the more generalized version you were thinking of.)

Code that gives special status to KeyboardInterrupt doesn't worry me at all. What I am wary of is code that is not written to be exception safe in the presence of unexpected exceptions (not just asynchronous ones). The example I was working on yesterday involved KeyboardInterrupts generated from an SSL socket in httplib's SSL readline method. A keyboard interrupt will result in data read from the socket and then stored in a local variable to be lost. Of course, this error wasn't very likely before I patched the SSL module to check for interrupts, but that is beside the point.

That's a problem that'll always remain. --Guido van Rossum (home page: http://www.python.org/~guido/)

Kevin Jacobs wrote:

On Thu, 12 Jun 2003, Guido van Rossum wrote:

...

...

In that past, there have been a few times where it would have been very convenient to terminate/signal another thread by posting an exception to it. Using KeyboardInterrupt to do so is okay, although it seems like something of a kludge.

This gets into the general discussion about whether threads should be stoppable.

True -- I am of the opinion that threads should be cooperatively stoppable, but not preemptively stoppable, if that makes sense. Asynchronous exceptions are the ideal way of implementing this, since the main interpreter loop is essentially already instrumented to handle them.

Exactly.

...

Also, this mechanism can *only* interrupt the main thread. (That's even true of the more generalized version you were thinking of.)

I wonder if that limitation would be liftable? Anybody interested in persuing this with me?

Ah.. that sounds familiar. I knew there was a reason I didn't pursue this avenue farther last time I poked through that code.

Same here. The code that checks for interrupts explicitly does nothing when called from a thread that's not the main thread. Naming: putting this into main sounds just right, but I would suggest to call it sys.interrupt() instead of sys.interrupt_main(). That way *if* we ever add the possibility to interrupt a non-main thread, sys.interrupt() could grow an optional argument specifying the thread. Just

Kevin Jacobs wrote:

...

...

Ah.. that sounds familiar. I knew there was a reason I didn't pursue this avenue farther last time I poked through that code.

Same here. The code that checks for interrupts explicitly does nothing when called from a thread that's not the main thread.

That chunk of code is very problematic. Some platforms/threading libraries will deliver a SIGINT to all threads, which can abort many system calls that are not trivially restarted/restartable. To deliver a KeyboardInterrupt only to the main thread is problematic, since the other threads may misinterpret the failure as something else (EOF, fatal error, or worse, operation succeeded).

I understand the desire to smooth over platform differences in Python, but I'm not sure that this one (signal delivery policy to threads) is a good idea. In fact, I'll try to find a real-world case where this kind of error occurs.

While the signal module is used in the implementation of PyErr_SetInterrupt(), I don't think signals play a part when interrupting threads: all that needs to be done is add a pending call that sets KeyboardInterrupt and returns -1. The problem is that the pending calls mechanism is for the main thread onle, and involves globals (mainly ceval.c's 'pendingcalls' and 'things_to_do'). So the first thing we'd need to look into is moving those globals to the PyThreadState struct (or to PyThreadState's dict object). (Btw. whatever we do, it won't interrupt blocking system calls, but I think that's fine. sys.interrupt() won't interrupt these either.) Just

(Btw. whatever we do, it won't interrupt blocking system calls, but I think that's fine. sys.interrupt() won't interrupt these either.)

In particular, this means that a thread blocked waiting for a lock won't be stopped until the lock is released by the thread that held the lock. If there's a deadlock, this may be never. So the mechanism won't help you to kill deadlocked threads. (This can be alleviated by using aquire() with a timeout everywhere; but the basic locks currently don't have that feature, they only have a non-blocking option. And of course code using timeouts is more complicated to write, and harder to read.) There's also a race condition in the language design that would need to be fixed before I'd like to have this feature. As you know, the proper idiom for using locks is: lock.acquire() try: ...critical section code... finally: lock.release() It's important that the acquire() call is outside the try block, because otherwise the release() call could be reached without the acquire() call having returned. But now if an asynchronous exception arrives *after* acquire() returns but *before* the try block is entered, the lock is never released! I've seen a proposal (I think it was in a paper presented at the second Little Languages conference at MIT, earlier this year) of a syntactic extension that could be used: initially: lock.acquire() try: ...critical section code... finally: lock.release() (this would block asynchronous signals during the initially: block) but that's a pretty heavy-handed solution. --Guido van Rossum (home page: http://www.python.org/~guido/)

Kevin Jacobs writes:

Why doesn't Python support the following fully general syntax for try-blocks:

try: foo() except blah: ... except: ... else: ... finally: ...

Will an exception, return, continue or break in an except: or else: block cause the finally: block to run? Bernhard -- Intevation GmbH http://intevation.de/ Sketch http://sketch.sourceforge.net/ MapIt! http://www.mapit.de/

This just reminded me of a question that has been floating around in my brain for a while. Why doesn't Python support the following fully general syntax for try-blocks:

try: foo() except blah: ... except: ... else: ... finally: ...

I keep having to write:

try: try: foo() except blah: ... except: ... else: ... finally: ...

At some early point in Python's development, it did support this. I decided to take it out for two reasons: (1) There were so many cases that I couldn't manage to create correct code; (2) the semantics aren't clear: does an exception raised in any of the except clauses jump to the finally clause or not? I'd say it should, but it may not be clear to users (because the except clauses don't apply to each other). It also complicates the bytecode generation: effectively, the compiler has to generate the bytecode equivalent to the second form. The second form makes this explicitly clear. Anyway, I don't think I've seen many too many cases where this would have saved an indentation level. --Guido van Rossum (home page: http://www.python.org/~guido/)

Part of the idea (or at least my idea) behind adding the do/with statement which was discussed a while ago was to fix this problem.

with lock: ... critical section ...

would ensure that lock.release() is called properly.

Right. That was also my idea (I think many people've had this idea :-). It has the advantage of being more generally useful than adding an initially clause to the try/finally statement. --Guido van Rossum (home page: http://www.python.org/~guido/)

[Kevin J]

That chunk of code is very problematic. Some platforms/threading libraries will deliver a SIGINT to all threads, which can abort many system calls that are not trivially restarted/restartable.

While I was worried about this when writing the original code (that's the only way to explain some of the things it does), I'm not sure if we really have to worry about such platforms any more; I certainly have never heard a report of such behavior. On Unix and Unixish systems, pthreads rule. Windows doesn't have real signals. Mac OS is now also Unix. PalmOS doesn't have threads. What other platforms are relevant?

To deliver a KeyboardInterrupt only to the main thread is problematic, since the other threads may misinterpret the failure as something else (EOF, fatal error, or worse, operation succeeded).

*If* the other threads indeed get their syscalls interrupted, this would indeed be a problem.

I understand the desire to smooth over platform differences in Python, but I'm not sure that this one (signal delivery policy to threads) is a good idea. In fact, I'll try to find a real-world case where this kind of error occurs.

I'd appreciate it if you found one. Nevertherless, I'm not sure if the weakening of semantics would make sense. Better to fix the offending platform (possibly by using pthreads :-). [Just]

...

Naming: putting this into main sounds just right, but I would suggest to call it sys.interrupt() instead of sys.interrupt_main(). That way *if* we ever add the possibility to interrupt a non-main thread, sys.interrupt() could grow an optional argument specifying the thread.

I Agree.

Hm... I like it better in the thread module, and there "thread.interrupt()" looks wrong, while "thread.interrupt_main()" makes sense. If and when the generalized functionality is implemented, I think the use cases will be so different that a different name for it won't be a big burden. I call YAGNI on the naming consideration. --Guido van Rossum (home page: http://www.python.org/~guido/)

Kevin Jacobs :

That chunk of code is very problematic. Some platforms/threading libraries will deliver a SIGINT to all threads, which can abort many system calls that are not trivially restarted/restartable.

But aren't we just talking about having one thread explicitly cause an asynchronous exception to be raised in another? That's a different thing from handling Unix signals. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

Guido van Rossum writes: [Jack Jansen]

...

+1 on the functionality. But: if it's so small why not simply put it in sys?

Hm, that's not bad. It should be called sys.interrupt_main() then.

Works for me! http://sourceforge.net/tracker/index.php?func=detail&aid=753733&group_id=5470&atid=305470 __ KBK

Guido van Rossum writes:

...

...

Hm, that's not bad. It should be called sys.interrupt_main() then.

Works for me!

http://sourceforge.net/tracker/index.php?func=detail&aid=753733&group_id=5470&atid=305470

Hm, but since this only makes sense if you have threads, shouldn't it be in the thread extension module? Also, you forgot the patch to idlelib/run.py. :-)

Oh, I did that when I tested sys.interrupt_main() :-) But not checked in until I saw whether the patch would stick.

If this makes sense to you, just check it in.

It sounds to me like thread.interrupt_main() is the way to go for now. That naming makes the limitation evident. If there should be a subsequent generalization to the implementation, it's no problem to update IDLE's run.py if thread.interrupt_main() gets deprecated. I'll post a revision to the patch. __ KBK

How should dummy_thread be modified? Something like

def interrupt_main(): raise KeyboardInterrupt

I don't think dummy_thread needs to support this API. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:

...

How should dummy_thread be modified? Something like

def interrupt_main(): raise KeyboardInterrupt

I don't think dummy_thread needs to support this API.

Well, it does now. =) I modified dummy_thread to raise KeyboardInterrupt after executing a function passed to start_new_thread if interrupt_main is called. I think this simulates threading a little better since, calling this you have no clue how long it will take for your code to keep executing so might as well let whatever is running finish before the "main" thread gets its exception. The only issue I see is someone calling interrupt_main and expecting an instant raise of KeyboardInterrupt if they are in the main thread. Damn, I need to modify the code again. [A few minutes later...] OK, done. If someone thinks this is overblown then just back the changes out but I figure the module might as well *try* to simulate threads as best as it can without getting too complex. -Brett