Two random and nearly unrelated ideas
While adding a blurb to Misc/NEWS about the change to the thread ticker and check interval, it occurred to me that perhaps Misc/NEWS would benefit from conversion to ReST format. You could pump an HTML version out to the website periodically. Second (also considered during the above edit), it would be nice to get rid of the ticker altogether in systems with proper signal support. On those platforms couldn't an alarm replace polling for the ticker? I know signals are tricky devils, but it still seems it would be a win if you could use it. You'd have to install a SIGALRM handler which would trip periodically. It would also have to keep track of any alarm handler the programmer installed. Just for the heck of it I recompiled ceval.c with the (--_Py_Ticker < 0) block ifdef'd out. Got a 1.7% increase in pystones over the now default checkinterval == 100 situation. Skip
On Tue, Sep 03, 2002 at 04:39:01PM -0500, Skip Montanaro wrote:
Second (also considered during the above edit), it would be nice to get rid of the ticker altogether in systems with proper signal support. On those platforms couldn't an alarm replace polling for the ticker?
Not before all all Python I/O calls are converted to be EINTR-safe. After running into some problems with I/O interrupted by signals I tried to fix it myself but it requires a lot of work in some of the hairiest places in the Python codebase. Oren
oren wrote:
Not before all all Python I/O calls are converted to be EINTR-safe.
After running into some problems with I/O interrupted by signals I tried to fix it myself but it requires a lot of work in some of the hairiest places in the Python codebase.
sounds like a good topic for a "here's what I learned when trying to fix this problem" PEP. </F>
Second (also considered during the above edit), it would be nice to get rid of the ticker altogether in systems with proper signal support. On those platforms couldn't an alarm replace polling for the ticker?
Not before all all Python I/O calls are converted to be EINTR-safe.
After running into some problems with I/O interrupted by signals I tried to fix it myself but it requires a lot of work in some of the hairiest places in the Python codebase.
Signals: just say no. It is impossible to write correct code in the presence of signals. --Guido van Rossum (home page: http://www.python.org/~guido/)
On Wed, Sep 04, 2002 at 07:44:32AM -0400, Guido van Rossum wrote:
Second (also considered during the above edit), it would be nice to get rid of the ticker altogether in systems with proper signal support. On those platforms couldn't an alarm replace polling for the ticker?
Not before all all Python I/O calls are converted to be EINTR-safe.
After running into some problems with I/O interrupted by signals I tried to fix it myself but it requires a lot of work in some of the hairiest places in the Python codebase.
Signals: just say no. It is impossible to write correct code in the presence of signals.
Wrapping all I/O calls with PyOS_ wrappers would be a good start. After that the wrappers can be modified to retry the call on EINTR. This should solve all the problems I have encountered with interference to Python code by signals. Any other problems I should be aware of? Oren
Signals: just say no. It is impossible to write correct code in the presence of signals.
Wrapping all I/O calls with PyOS_ wrappers would be a good start.
And what should those wrappers do?
After that the wrappers can be modified to retry the call on EINTR.
But that's not always what you want to happen! E.g. if an app is blocked on a read and uses an alarm to bail out of the read.
This should solve all the problems I have encountered with interference to Python code by signals. Any other problems I should be aware of?
There's no way to sufficiently test a program that uses signals. The signal handler cannot touch *any* data, which makes it pretty useless. --Guido van Rossum (home page: http://www.python.org/~guido/)
On Wed, Sep 04, 2002 at 09:25:01AM -0400, Guido van Rossum wrote:
After that the wrappers can be modified to retry the call on EINTR.
But that's not always what you want to happen! E.g. if an app is blocked on a read and uses an alarm to bail out of the read.
If I use a module that spawns an external process and uses SIGCHLD to be informed of its termination why should my innocent code that just reads lines from a file suddenly break? In C I can at least restart the operation after an EINTR but file.readline cannot even be properly restarted because the buffering and file position is all messed up. The example you gave of bailing out of a read with a signal can be done using other techniques such as non-blocking I/O (which is, IMHO, a much cleaner way to do it). Getting an notification of a child process terminating or other asynchronous events can only be done using signals and is currently dangerous because it will break code using I/O.
interference to Python code by signals. Any other problems I should be aware of?
There's no way to sufficiently test a program that uses signals. The signal handler cannot touch *any* data, which makes it pretty useless.
In order to be useful a signal handler needs to be able to set one bit. The next time the ticker expires this bit will be checked. If an I/O operation was interrupted the Python signal handler can be executed immediately from the wrapper. When it returns the wrapper will resume the interrupted operation. Oren I/O, I/O, it's off to work we go... The seven dwarfs
If I use a module that spawns an external process and uses SIGCHLD to be informed of its termination why should my innocent code that just reads lines from a file suddenly break? In C I can at least restart the operation after an EINTR but file.readline cannot even be properly restarted because the buffering and file position is all messed up.
I have never understood why a child dying should send a signal. You can poll for the child with waitpid() instead. But if you have a suggestion for how to fix this particular issue, I'd be happy to look it over, since this *is* something some people do.
The example you gave of bailing out of a read with a signal can be done using other techniques such as non-blocking I/O (which is, IMHO, a much cleaner way to do it).
Yes.
Getting an notification of a child process terminating or other asynchronous events can only be done using signals and is currently dangerous because it will break code using I/O.
See above. I see half your point; people wanting this tend to use signals and it causes breakage.
interference to Python code by signals. Any other problems I should be aware of?
There's no way to sufficiently test a program that uses signals. The signal handler cannot touch *any* data, which makes it pretty useless.
In order to be useful a signal handler needs to be able to set one bit. The next time the ticker expires this bit will be checked.
OK.
If an I/O operation was interrupted the Python signal handler can be executed immediately from the wrapper. When it returns the wrapper will resume the interrupted operation.
Is calling the Python signal handler from the wrapper always safe? What if the Python signal handler e.g. closes the file or reads from it? --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum
I have never understood why a child dying should send a signal. You can poll for the child with waitpid() instead.
Because child termination might not be the only thing you want to wait for. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
On Wed, Sep 04, 2002 at 04:05:22PM -0400, Guido van Rossum wrote:
If I use a module that spawns an external process and uses SIGCHLD to be informed of its termination why should my innocent code that just reads lines from a file suddenly break? In C I can at least restart the operation after an EINTR but file.readline cannot even be properly restarted because the buffering and file position is all messed up.
I have never understood why a child dying should send a signal. You can poll for the child with waitpid() instead.
You're assuming too much about the structure of the program using child processes. The code that starts the child process may not be in control of the Python program counter by the time it ends. It's useful to be able to leave a signal handler to clean up the zombie process by waitpid().
But if you have a suggestion for how to fix this particular issue, I'd be happy to look it over, since this *is* something some people do.
Of course people do it - it's documented and it works. Signal handling may have had some historical problems on some Unixes but I've never had any problem with it under Linux. My previous messages more or less outline my suggestion. I'll write a better summary.
Getting an notification of a child process terminating or other asynchronous events can only be done using signals and is currently dangerous because it will break code using I/O.
See above. I see half your point; people wanting this tend to use signals and it causes breakage.
Polling is not what I'd call "getting notification of asynchronous events". If it causes breakage it could be because people either use it incorrectly or the signal support on the underlying system is broken. In Linux it isn't broken. If it's broken on other Python platforms I don't see why it shouldn't be well-supported on the platforms that aren't. Has anyone here actually tried to use signal.signal ?
interference to Python code by signals. Any other problems I should be aware of?
There's no way to sufficiently test a program that uses signals. The signal handler cannot touch *any* data, which makes it pretty useless.
In order to be useful a signal handler needs to be able to set one bit. The next time the ticker expires this bit will be checked.
OK.
If an I/O operation was interrupted the Python signal handler can be executed immediately from the wrapper. When it returns the wrapper will resume the interrupted operation.
Is calling the Python signal handler from the wrapper always safe? What if the Python signal handler e.g. closes the file or reads from it?
Code in signal handlers is executed at some arbitrary point in the program and the programmer should be aware of this and only do so simple things like setting a flag or appending to a list. Oren
I have never understood why a child dying should send a signal. You can poll for the child with waitpid() instead.
You're assuming too much about the structure of the program using child processes. The code that starts the child process may not be in control of the Python program counter by the time it ends. It's useful to be able to leave a signal handler to clean up the zombie process by waitpid().
I admit that I hate signals so badly that whenever I needed to wait for a child to finish I would always structure the program around this need (even when coding in C).
But if you have a suggestion for how to fix this particular issue, I'd be happy to look it over, since this *is* something some people do.
Of course people do it - it's documented and it works.
Barely. This thread started when you pointed out the problems with using signals. I've always been reluctant about the fact that we had a signal module at all -- it's not portable (no non-Unix system supports it well), doesn't interact well with threads, etc., etc.; however, C programmers have demanded some sort of signal support and I caved in long ago when someone contributed a reasonable approach. I don't regret it like lambda, but I think it should only be used by people who really know about the caveats.
See above. I see half your point; people wanting this tend to use signals and it causes breakage.
Polling is not what I'd call "getting notification of asynchronous events". If it causes breakage it could be because people either use it incorrectly or the signal support on the underlying system is broken. In Linux it isn't broken. If it's broken on other Python platforms I don't see why it shouldn't be well-supported on the platforms that aren't.
I meant in Python. The I/O problems make signals hard to use.
Has anyone here actually tried to use signal.signal ?
Yes.
Code in signal handlers is executed at some arbitrary point in the program and the programmer should be aware of this and only do so simple things like setting a flag or appending to a list.
Unfortunately the mechanism doesn't enforce this. I wish we could invent a Python signal API that only lets you do one of these simple things. --Guido van Rossum (home page: http://www.python.org/~guido/)
On Thu, Sep 05, 2002 at 11:01:14AM -0400, Guido van Rossum wrote:
I have never understood why a child dying should send a signal. You can poll for the child with waitpid() instead.
You're assuming too much about the structure of the program using child processes. The code that starts the child process may not be in control of the Python program counter by the time it ends. It's useful to be able to leave a signal handler to clean up the zombie process by waitpid().
I admit that I hate signals so badly that whenever I needed to wait for a child to finish I would always structure the program around this need (even when coding in C).
Ummm... if you really hate signals that much perhaps you to step aside from this particular discussion? Naturally, you will get to pronounce on the results that come out of it (if any ;-) Westley: No, no. We have already succeeded. I mean, what are the three terrors of the fire swamp? One, the flame spurt - no problem - there's a popping sound preceding each. We can avoid that. Two, the lightning sand which you were clever enough to discover what that looks like, so in the future we can avoid that too. (from "The Princess Bride" by William Goldman) So what are the three problems of signals? One - what calls are allowed by the platform inside a signal handler. No problem. Nobody suggested actually executing Python code inside a signal handler so we don't need to be worried about user code. The C handler doesn't call anything unusual, just sets flags. This should work on all platforms. Two - Interruptible system calls. If all Python I/O calls are wrapped inside restarting wrappers this should be solved. If the system's libc wraps them it can be disabled by SA_RESTART (posix) or siginterrupt (BSD). On some systems read and recv return a short buffer instead of EINTR. This can be safely ignored because it only happens for pipes and sockets where this is a valid result. AFAIR it's guaranteed not to happen on regular files so we won't be tricked into believing they reached EOF. Are there any systems where system calls are interruptible but not restartable in any way without data loss? Three - Threads playing "who gets the signal". The Python signal module has a hack that appears to work on all relevant platform - ignore the signal if getpid() isn't the main thread. Oren Buttercup: Westley, what about the R.O.U.S.'s? Westley: Rodents Of Unusual Size? I don't think they exist. ...
I admit that I hate signals so badly that whenever I needed to wait for a child to finish I would always structure the program around this need (even when coding in C).
Ummm... if you really hate signals that much perhaps you to step aside from this particular discussion? Naturally, you will get to pronounce on the results that come out of it (if any ;-)
Why? I don't think hating signals disqualifies me from understanding their problems.
So what are the three problems of signals?
One - what calls are allowed by the platform inside a signal handler. No problem. Nobody suggested actually executing Python code inside a signal handler so we don't need to be worried about user code. The C handler doesn't call anything unusual, just sets flags. This should work on all platforms.
Two - Interruptible system calls. If all Python I/O calls are wrapped inside restarting wrappers this should be solved.
I asked what the Python code called by the wrapper when a signal arrives is allowed to do (e.g. close the file?). If you replied to that, I missed it.
If the system's libc wraps them it can be disabled by SA_RESTART (posix) or siginterrupt (BSD). On some systems read and recv return a short buffer instead of EINTR.
This latter sentence shows that you don't understand signals, or you're being very sloppy. You get *either* a short buffer *or* EINTR depending on whether some data was already transferred to user space.
This can be safely ignored because it only happens for pipes and sockets where this is a valid result. AFAIR it's guaranteed not to happen on regular files so we won't be tricked into believing they reached EOF.
I don't believe that a short read on a regular file can be used reliably to infer EOF anyway. The file could be growing while we read.
Are there any systems where system calls are interruptible but not restartable in any way without data loss?
Not AFAIK.
Three - Threads playing "who gets the signal". The Python signal module has a hack that appears to work on all relevant platform - ignore the signal if getpid() isn't the main thread.
Doesn't that make signals unreliable? What if thread 4 has forked a child, and the child exist? Won't the SIGCHLD be sent to thread 4? AFAIK there's no standard for this, or if there is, not all systems comply. --Guido van Rossum (home page: http://www.python.org/~guido/)
On Thu, Sep 05, 2002 at 04:46:27PM -0400, Guido van Rossum wrote:
I admit that I hate signals so badly that whenever I needed to wait for a child to finish I would always structure the program around this need (even when coding in C).
Ummm... if you really hate signals that much perhaps you to step aside from this particular discussion? Naturally, you will get to pronounce on the results that come out of it (if any ;-)
Why? I don't think hating signals disqualifies me from understanding their problems.
In the past I have disqualified myself from making technical decisions on issues where I have been burned and knew that my opinion would be a calm rational decision.
I asked what the Python code called by the wrapper when a signal arrives is allowed to do (e.g. close the file?). If you replied to that, I missed it.
Anything that a Python thread is allowed to do without grabbing a lock, i.e. anything that involves only exclusive resources or atomic Python operations on shared resources like setting a variable (but not read-modify-write). A signal handler is also allowed to raise an exception that will get delivered to the main thread.
If the system's libc wraps them it can be disabled by SA_RESTART (posix) or siginterrupt (BSD). On some systems read and recv return a short buffer instead of EINTR.
This latter sentence shows that you don't understand signals, or you're being very sloppy. You get *either* a short buffer *or* EINTR depending on whether some data was already transferred to user space.
Did I say that you get *both* a short buffer *and* EINTR? What I meant is that it's really quite simple - if errno==EINTR I retry and if I get a short buffer I continue from whatever I got and ask for the remainder and this should work regardless of the differences in behavior between different systems, sockets and files, etc.
This can be safely ignored because it only happens for pipes and sockets where this is a valid result. AFAIR it's guaranteed not to happen on regular files so we won't be tricked into believing they reached EOF.
I don't believe that a short read on a regular file can be used reliably to infer EOF anyway. The file could be growing while we read.
You're right, only a zero result on read should be interpreted as EOF, not a short result. I got confused by fread where a short read does mark an end of file condition. I don't see how the growing file case is relevant, though.
Three - Threads playing "who gets the signal". The Python signal module has a hack that appears to work on all relevant platform - ignore the signal if getpid() isn't the main thread.
Doesn't that make signals unreliable? What if thread 4 has forked a child, and the child exist? Won't the SIGCHLD be sent to thread 4? AFAIK there's no standard for this, or if there is, not all systems comply.
I've never actually tried this one. I just went by the comments in signalmodule.c which claim that this works for all cases of how different implementations deliver signals to threads. I guess that was a bit hasty. Oren
On Thu, 5 Sep 2002, Oren Tirosh wrote:
So what are the three problems of signals?
Two - Interruptible system calls. If all Python I/O calls are wrapped inside restarting wrappers this should be solved. If the system's libc wraps them it can be disabled by SA_RESTART (posix) or siginterrupt (BSD). On some systems read and recv return a short buffer instead of EINTR. This can be safely ignored because it only happens for pipes and sockets where this is a valid result. AFAIR it's guaranteed not to happen on regular files so we won't be tricked into believing they reached EOF. Are there any systems where system calls are interruptible but not restartable in any way without data loss?
I don't see any guarantee against short reads in my documentation (Linux, HP-UX); indeed both state explicitly that only a 0 return from read() indicates EOF. /Paul
On donderdag, september 5, 2002, at 05:01 , Guido van Rossum wrote:
Code in signal handlers is executed at some arbitrary point in the program and the programmer should be aware of this and only do so simple things like setting a flag or appending to a list.
Unfortunately the mechanism doesn't enforce this. I wish we could invent a Python signal API that only lets you do one of these simple things.
Could we connect signals to semaphores or locks or something
like that? That would allow you to do the two things that i
think are worth doing in a signal handler: setting a flag and/or
making some other part of the code wake up.
Only problem is that for completeness you would really want to
wire up select-like functionality too, so that you could really
have a single waiting mechanism.
--
- Jack Jansen
Could we connect signals to semaphores or locks or something like that? That would allow you to do the two things that i think are worth doing in a signal handler: setting a flag and/or making some other part of the code wake up.
But that mixes signals with threads, which is even more poorly standardized than signals in general. --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
Could we connect signals to semaphores or locks or something like that? That would allow you to do the two things that i think are worth doing in a signal handler: setting a flag and/or making some other part of the code wake up.
But that mixes signals with threads, which is even more poorly standardized than signals in general.
Python can open a pipe to itself. When a signal arrives, write a character on the pipe in addition to setting a flag. Then select() on the pipe. I doubt this is worth the effort, though. Neal
[Neal Norwitz]
Python can open a pipe to itself. When a signal arrives, write a character on the pipe in addition to setting a flag. Then select() on the pipe.
Of course you meant to say it should do WaitForSingleObject(), so that this scheme is portable <wink>.
I doubt this is worth the effort, though.
Few things are.
On Fri, Sep 06, 2002 at 05:09:16PM +0200, Jack Jansen wrote:
Could we connect signals to semaphores or locks or something like that? That would allow you to do the two things that i think are worth doing in a signal handler: setting a flag and/or making some other part of the code wake up.
Signal handlers and locks don't mix well. A signal handler can't grab a lock. The signal handler can't wait for the lock to be released because it has interrupted the code holding it. The traditional way this has been handled is with a global "interrupt enable" flag. Just like the good old days of 8 bit micros and DOS when any application could clear the interrupt flag :-) If Queue.Queue sets up a signal critical section as well as getting the queue lock a signal could write to a Queue and wake up a thread waiting on the other end.
Only problem is that for completeness you would really want to wire up select-like functionality too, so that you could really have a single waiting mechanism.
If the program uses select as the central dispatcher you can set up a pipe. The signal handler writes to one end and the other end is listed in the select socket map. It's a simple way to handle an occasional event like a child process dying or a SIGHUP telling you to reload the configuration file. Do you want to use signals for more intensive tasks like asynchronous I/O? Oren
On Fri, Sep 06, 2002 at 07:54:49PM +0300, Oren Tirosh wrote:
Signal handlers and locks don't mix well. A signal handler can't grab a lock. The signal handler can't wait for the lock to be released because it has interrupted the code holding it. The traditional way this has been handled is with a global "interrupt enable" flag. Just like the good old days of 8 bit micros and DOS when any application could clear the interrupt flag :-)
If Queue.Queue sets up a signal critical section as well as getting the queue lock a signal could write to a Queue and wake up a thread waiting on the other end.
Would this be an appropriate place to complain about how KeyboardInterrupt won't wake up a thread stuck waiting on a Queue? zw
Would this be an appropriate place to complain about how KeyboardInterrupt won't wake up a thread stuck waiting on a Queue?
No, unless you have a real proposal on how to fix it (not just a vague idea -- we've all had those, and they don't work). Working code or shut up. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)
On Fri, Sep 06, 2002 at 01:53:22PM -0400, Guido van Rossum wrote:
Would this be an appropriate place to complain about how KeyboardInterrupt won't wake up a thread stuck waiting on a Queue?
No, unless you have a real proposal on how to fix it (not just a vague idea -- we've all had those, and they don't work). Working code or shut up. :-)
Fair enough. The underlying problem is that KeyboardInterrupt does not abort acquire() called on a thread lock. This is only noticeable when it was the main thread that called acquire -- if it's some other thread, the KeyboardInterrupt will still be delivered to the main thread. Compare the behavior of these two test programs: -- test1.py -- import time, thread lock = thread.allocate_lock() lock.acquire() def child_thread(): print "Acquiring lock" lock.acquire() print "Have lock (can't happen)" lock.release() thread.start_new_thread(child_thread, ()) print "Hit ^C now" time.sleep(3600) -- test2.py -- import time, thread lock = thread.allocate_lock() def child_thread(): print "Acquiring lock" lock.acquire() print "Have lock" time.sleep(3600) lock.release() thread.start_new_thread(child_thread, ()) time.sleep(1) # give child a chance to acquire lock print "Hit ^C now" lock.acquire() I'm going to look only at the pthread-based thread support; presumably similar changes to the ones I will propose, need to be made to the others. There are two cases of PyThread_acquire_lock in thread_pthread.h: using semaphores, and using condition variables. Let's look at the condition variable one first: /* mut must be locked by me -- part of the condition * protocol */ status = pthread_mutex_lock( &thelock->mut ); CHECK_STATUS("pthread_mutex_lock[2]"); while ( thelock->locked ) { status = pthread_cond_wait(&thelock->lock_released, &thelock->mut); CHECK_STATUS("pthread_cond_wait"); } thelock->locked = 1; status = pthread_mutex_unlock( &thelock->mut ); Naively, we'd like to shove a check of PyOS_InterruptOccurred in that loop so we can bail out if it's true. It is part of the spec for pthread_cond_wait that any signal which is handled (as SIGINT is) will not interrupt its execution. So in order to get a chance to check for interrupts we need to change this to a repeated timed wait, like so: while ( thelock->locked && !interrupted ) { timeout.tv_sec = time(0) + 1; status = pthread_cond_timedwait(&thelock->lock_released, &thelock->mut, &timeout); if (status != ETIMEDOUT) CHECK_STATUS("pthread_cond_wait"); interrupted = PyOS_InterruptOccurred(); } thelock->locked = 1; status = pthread_mutex_unlock( &thelock->mut ); Then we do a bit of fiddling in the return path to reset the interrupt flag and make sure the caller sees a failure. In the semaphore case, life is theoretically simpler: there is no mutex, and sem_wait is interrupted by a handled signal, assuming SA_RESTART was not set for that signal (which it isn't, in Python). do { if (waitflag) status = fix_status(sem_wait(thelock)); else status = fix_status(sem_trywait(thelock)); } while (status == EINTR); /* Retry if interrupted by a signal */ becomes do { if (waitflag) status = fix_status(sem_wait(thelock)); else status = fix_status(sem_trywait(thelock)); if (status == EINTR && PyOS_InterruptOccurred()) goto interrupted; } while (status == EINTR); /* Retry if interrupted by a signal */ ... interrupted: PyErr_SetInterrupt(); dprintf(("PyThread_acquire_lock(%p, %d) interrupted by user\n", lock, waitflag)); return 0; However, the Linux semaphore implementation is buggy and will not actually return EINTR from sem_wait, ever. I'll take this up with the libc maintainers; at the Python level, the thing to do is assume it works. Hence, the appended patch. (While I was at it I fixed CHECK_STATUS so that it actually prints the relevant system error, instead of whatever junk happens to be in errno.) zw =================================================================== Index: thread_pthread.h --- thread_pthread.h 17 Mar 2002 17:19:00 -0000 2.40 +++ thread_pthread.h 6 Sep 2002 20:51:31 -0000 @@ -128,7 +128,12 @@ typedef struct { pthread_mutex_t mut; } pthread_lock; -#define CHECK_STATUS(name) if (status != 0) { perror(name); error = 1; } +#define CHECK_STATUS(name) do { \ + if (status != 0) { \ + fprintf(stderr, "%s: %s\n", name, strerror(status)); \ + error = 1; \ + } \ +} while (0) /* * Initialization. @@ -387,6 +392,8 @@ PyThread_acquire_lock(PyThread_type_lock status = fix_status(sem_wait(thelock)); else status = fix_status(sem_trywait(thelock)); + if (status == EINTR && PyOS_InterruptOccurred()) + goto interrupted; } while (status == EINTR); /* Retry if interrupted by a signal */ if (waitflag) { @@ -399,6 +406,12 @@ PyThread_acquire_lock(PyThread_type_lock dprintf(("PyThread_acquire_lock(%p, %d) -> %d\n", lock, waitflag, success)); return success; + + interrupted: + PyErr_SetInterrupt(); + dprintf(("PyThread_acquire_lock(%p, %d) interrupted by user\n", + lock, waitflag)); + return 0; } void @@ -472,8 +485,10 @@ int PyThread_acquire_lock(PyThread_type_lock lock, int waitflag) { int success; + int interrupted = 0; pthread_lock *thelock = (pthread_lock *)lock; int status, error = 0; + struct timespec timeout; dprintf(("PyThread_acquire_lock(%p, %d) called\n", lock, waitflag)); @@ -491,10 +506,15 @@ PyThread_acquire_lock(PyThread_type_lock * protocol */ status = pthread_mutex_lock( &thelock->mut ); CHECK_STATUS("pthread_mutex_lock[2]"); - while ( thelock->locked ) { - status = pthread_cond_wait(&thelock->lock_released, - &thelock->mut); - CHECK_STATUS("pthread_cond_wait"); + timeout.tv_nsec = 0; + while ( thelock->locked && !interrupted ) { + timeout.tv_sec = time(0) + 1; + status = pthread_cond_timedwait(&thelock->lock_released, + &thelock->mut, + &timeout); + if (status != ETIMEDOUT) + CHECK_STATUS("pthread_cond_wait"); + interrupted = PyOS_InterruptOccurred(); } thelock->locked = 1; status = pthread_mutex_unlock( &thelock->mut ); @@ -502,6 +522,10 @@ PyThread_acquire_lock(PyThread_type_lock success = 1; } if (error) success = 0; + if (interrupted) { + PyErr_SetInterrupt(); + success = 0; + } dprintf(("PyThread_acquire_lock(%p, %d) -> %d\n", lock, waitflag, success)); return success; }
>> On those platforms couldn't an alarm replace polling for the ticker? Oren> Not before all all Python I/O calls are converted to be Oren> EINTR-safe. Ah, yes. Thanks for pointing out that little stumbling block... Skip
While adding a blurb to Misc/NEWS about the change to the thread ticker and check interval, it occurred to me that perhaps Misc/NEWS would benefit from conversion to ReST format. You could pump an HTML version out to the website periodically.
Nice idea. How much additional mark-up would this add to quote the occasional reST meta-character? Can you convert a section for test and show me?
Second (also considered during the above edit), it would be nice to get rid of the ticker altogether in systems with proper signal support. On those platforms couldn't an alarm replace polling for the ticker? I know signals are tricky devils, but it still seems it would be a win if you could use it. You'd have to install a SIGALRM handler which would trip periodically. It would also have to keep track of any alarm handler the programmer installed.
-1,000,000. --Guido van Rossum (home page: http://www.python.org/~guido/)
[Skip]
While adding a blurb to Misc/NEWS about the change to the thread ticker and check interval, it occurred to me that perhaps Misc/NEWS would benefit from conversion to ReST format. You could pump an HTML version out to the website periodically.
I have the Docutils site auto-regenerated via a small cron script. Any time any of the source text files change, within an hour the site reflects the change. It makes site maintenance easy. (BTW, Skip, thanks for the bug report. I'll be looking into it ASAP.) [Guido]
Nice idea. How much additional mark-up would this add to quote the occasional reST meta-character?
Very little, depending on the desired effect. The extreme case would be if you want to mark up everything possible. The result may look too busy in the source text form though, especially because there are so many Python identifiers, expressions, code snippets, and file names that *could* be marked up. It's a trade-off. The nice thing is that Misc/NEWS is already almost valid reStrucuturedText (which shouldn't be surprising, since reStrucuturedText is based on common usage). In fact, most (if not all) of the standalone text files are almost there: README, PLAN.txt, etc. It wouldn't be much work to bring them up to spec. Here are the areas of Misc/NEWS that would require editing: * Sections: The two-line titles aren't supported. Either they should be combined into one line, or the "Release date" line should become part of the section body. Either:: What's New in Python 2.2 final? Release date: 21-Dec-2001 ========================================================== or:: What's New in Python 2.2 final? =============================== Release date: 21-Dec-2001 * Subsections (like "Core and builtins", "Library", "Extension modules", etc.): These could be made into true subsections by underlining them with dashes (and changing to title case):: Core and Builtins ----------------- I notice that there are many headers for empty subsections (such as "Tools/Demos" and "Build" in "What's New in Python 2.2 final?"). Should they be removed? * Inline literals (filenames, identifiers, expressions and code snippets): Surround with double-backquotes to get monospaced, uninterpreted text (like HTML TT tags). There are so many of these that it may be best to be selective. * Literal blocks: Example code should be indented and prefaced with double-colons ("::" at the end of the preceding paragraph). Doctest blocks (interactive sessions, begin with ">>> " and end with a blank line) don't need this, although it wouldn't hurt.
Can you convert a section for test and show me?
I'll be happy to help. Hmm. Looking at the 2.2.1 Misc/NEWS file, I see sections for 2.2.1 final, 2.2.1c2, etc., but they're missing from the CVS Misc/NEWS file. Is this normal because of separate development branches or is something amiss? Following is a converted section from the current Misc/NEWS. Minimally marked up:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
What's New in Python 2.3 alpha 1? ================================= XXX Release date: DD-MMM-2002 XXX Type/class unification and new-style classes -------------------------------------------- - Assignment to __class__ is disallowed if either the old and the new class is a statically allocated type object (such as defined by an extension module). This prevents anomalies like ``2 .__class__ = bool``. - New-style object creation and deallocation have been sped up significantly; they are now faster than classic instance creation and deallocation. - The __slots__ variable can now mention "private" names, and the right thing will happen (e.g. ``__slots__ = ["__foo"]``). - The built-ins slice() and buffer() are now callable types. The types classobj (formerly class), code, function, instance, and instancemethod (formerly instance-method), which have no built-in names but are accessible through the types module, are now also callable. The type dict-proxy is renamed to dictproxy. - Cycles going through the __class__ link of a new-style instance are now detected by the garbage collector. - Classes using __slots__ are now properly garbage collected. [SF bug 519621] - Tightened the __slots__ rules: a slot name must be a valid Python identifier. - The constructor for the module type now requires a name argument and takes an optional docstring argument. Previously, this constructor ignored its arguments. As a consequence, deriving a class from a module (not from the module type) is now illegal; previously this created an unnamed module, just like invoking the module type did. [SF bug 563060] - A new type object, 'basestring', is added. This is a common base type for 'str' and 'unicode', and can be used instead of ``types.StringTypes``, e.g. to test whether something is "a string": ``isinstance(x, basestring)`` is True for Unicode and 8-bit strings. This is an abstract base class and cannot be instantiated directly. - Changed new-style class instantiation so that when C's __new__ method returns something that's not a C instance, its __init__ is not called. [SF bug #537450] - Fixed super() to work correctly with class methods. [SF bug #535444] - If you try to pickle an instance of a class that has __slots__ but doesn't define or override __getstate__, a TypeError is now raised. This is done by adding a bozo __getstate__ to the class that always raises TypeError. (Before, this would appear to be pickled, but the state of the slots would be lost.) <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< Maximally marked up:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
What's New in Python 2.3 alpha 1?
=================================
XXX Release date: DD-MMM-2002 XXX
Type/class unification and new-style classes
--------------------------------------------
- Assignment to ``__class__`` is disallowed if either the old and the
new class is a statically allocated type object (such as defined by
an extension module). This prevents anomalies like ``2 .__class__ =
bool``.
- New-style object creation and deallocation have been sped up
significantly; they are now faster than classic instance creation
and deallocation.
- The ``__slots__`` variable can now mention "private" names, and the
right thing will happen (e.g. ``__slots__ = ["__foo"]``).
- The built-ins ``slice()`` and ``buffer()`` are now callable types.
The types classobj (formerly class), code, function, instance, and
instancemethod (formerly instance-method), which have no built-in
names but are accessible through the ``types`` module, are now also
callable. The type dict-proxy is renamed to dictproxy.
- Cycles going through the ``__class__`` link of a new-style instance
are now detected by the garbage collector.
- Classes using ``__slots__`` are now properly garbage collected.
[SF bug 519621]
- Tightened the ``__slots__`` rules: a slot name must be a valid
Python identifier.
- The constructor for the module type now requires a name argument and
takes an optional docstring argument. Previously, this constructor
ignored its arguments. As a consequence, deriving a class from a
module (not from the module type) is now illegal; previously this
created an unnamed module, just like invoking the module type did.
[SF bug 563060]
- A new type object, ``basestring``, is added. This is a common base
type for ``str`` and ``unicode``, and can be used instead of
``types.StringTypes``, e.g. to test whether something is "a string":
``isinstance(x, basestring)`` is ``True`` for Unicode and 8-bit
strings. This is an abstract base class and cannot be instantiated
directly.
- Changed new-style class instantiation so that when C's ``__new__``
method returns something that's not a C instance, its ``__init__``
is not called. [SF bug #537450]
- Fixed ``super()`` to work correctly with class methods. [SF bug #535444]
- If you try to pickle an instance of a class that has ``__slots__``
but doesn't define or override ``__getstate__``, a ``TypeError`` is
now raised. This is done by adding a bozo ``__getstate__`` to the
class that always raises ``TypeError``. (Before, this would appear
to be pickled, but the state of the slots would be lost.)
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
--
David Goodger
participants (11)
-
David Goodger -
Fredrik Lundh -
Greg Ewing -
Guido van Rossum -
Jack Jansen -
Neal Norwitz -
Oren Tirosh -
Paul Svensson -
Skip Montanaro -
Tim Peters -
Zack Weinberg