Hello. As this is my first post I will try to introduce myself as requested in the welcome email. If you aren't interested in my person, just continue reading at the next paragraph. I'm a student from Vienna/Austria. I attend what would match high school in the United States. I have been writing Python for about 3 years now and have just began to dig into the implementation of CPython. Now to my real question. I have noticed that PyThread_acquire_lock swallows all signals with pthread using sems. Looking at the source code it appears that this is intended, but I cannot see the reason for that. It seems the pthread sem implementation is the only one doing so. Can any of you tell me the reason why it swallows signals? I have already prepared a patch that introduces a new _noswallow function if sem are used and uses this in threadmodule.c. But if there isn't any real reason for it to swallow exceptions, I think it would be wisest to change that. Best regards, Florian Mayer
Now to my real question. I have noticed that PyThread_acquire_lock swallows all signals with pthread using sems. Looking at the source code it appears that this is intended, but I cannot see the reason for that. It seems the pthread sem implementation is the only one doing so. Can any of you tell me the reason why it swallows signals?
Can you please explain what you mean by "swallowing"? What is the sequence of actions triggering the case you are talking about, what happens, and what do you expect to happen instead? Also, how would you fix this (in principle, not what the specific patch would look like)? Regards, Martin
Martin v. Löwis wrote:
Can you please explain what you mean by "swallowing"? The signals don't get through when acquiring the lock. What is the sequence of actions triggering the case you are talking about, what happens, and what do you expect to happen instead? Create a Lock object, call .acquire and try to press Ctrl-C, the keyboardinterrupt is swallowed Also, how would you fix this (in principle, not what the specific patch would look like)? Remove the loop that explicitly does this in a new function and use that one in threadmodule.c for lock.acquire.
Regards, Martin Best regards, Florian
PS: Sorry for sending this message twice.
Martin v. Löwis wrote:
Can you please explain what you mean by "swallowing"? The signals don't get through when acquiring the lock. What is the sequence of actions triggering the case you are talking about, what happens, and what do you expect to happen instead? Easiest way is to create a Queue.Queue and call .get() on it and try to Ctrl-C it. Also, how would you fix this (in principle, not what the specific patch would look like)? Remove the loop that explicitly does this in a new function and use that one in threadmodule.c for lock.acquire.
Regards, Martin Best regards, Florian
PS: Excuse me, I messed up things once again.
I'll do my best to try and explain/contribute, but please feel free to correct anything I get wrong. I believe the "swallowing" he's referring to is the ignoring of errno EINTR. I don't think that's the correct place to handle signals to begin with- why not just use the signal module to deal with such a scenario? http://docs.python.org/dev/library/signal.html#module-signal AFAIK, ignoring EINTR doesn't preclude the calling of signal handlers. There are many functions that don't return this value anymore, making them reentrant. I remember a number of years ago when it wasn't part of any standard to return EINTR or not, and so the only way to provide consistent behavior was to ignore it and loop. I'm not sure if that is still the case. A great example is reading from a socket. Whether or not it can be interrupted depends on the platform, so catching Ctrl+C often requires a timeout loop. Also, remember that signals are asynchronous in the sense that they are handled outside the normal execution flow of a program. Checking for EINTR probably isn't the best way to determine if a signal has been sent to the program. Cheers, Phillip On Sat, Jun 27, 2009 at 3:58 PM, Florian Mayer <flormayer@aim.com> wrote:
Martin v. Löwis wrote:
Can you please explain what you mean by "swallowing"?
The signals don't get through when acquiring the lock.
What is the sequence of actions triggering the case you are talking about, what happens, and what do you expect to happen instead?
Easiest way is to create a Queue.Queue and call .get() on it and try to Ctrl-C it.
Also, how would you fix this (in principle, not what the specific patch would look like)?
Remove the loop that explicitly does this in a new function and use that one in threadmodule.c for lock.acquire.
Regards, Martin
Best regards, Florian
PS: Excuse me, I messed up things once again.
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/phillip.sitbon%2Bpython-de...
AFAIK, ignoring EINTR doesn't preclude the calling of signal handlers.
This is my understanding as well - so I don't think Python actually "swallows" the signal.
A great example is reading from a socket. Whether or not it can be interrupted depends on the platform, so catching Ctrl+C often requires a timeout loop.
Also, remember that signals are asynchronous in the sense that they are handled outside the normal execution flow of a program. Checking for EINTR probably isn't the best way to determine if a signal has been sent to the program.
I think it would be reasonable to support "asynchronous" exceptions, and Python supports SIGINT fairly well most of the time. It might be possible to support keyboard interrupts throughout the system, but changing Python to do so could also cause incompatibilities. So any change must be done with greatest care, but simultaneously, should also try to arrange to cover all cases. Regards, Martin
On Mon, Jun 29, 2009 at 2:28 PM, "Martin v. Löwis"<martin@v.loewis.de> wrote:
AFAIK, ignoring EINTR doesn't preclude the calling of signal handlers.
This is my understanding as well - so I don't think Python actually "swallows" the signal.
A great example is reading from a socket. Whether or not it can be interrupted depends on the platform, so catching Ctrl+C often requires a timeout loop.
Also, remember that signals are asynchronous in the sense that they are handled outside the normal execution flow of a program. Checking for EINTR probably isn't the best way to determine if a signal has been sent to the program.
I think it would be reasonable to support "asynchronous" exceptions, and Python supports SIGINT fairly well most of the time.
It might be possible to support keyboard interrupts throughout the system, but changing Python to do so could also cause incompatibilities. So any change must be done with greatest care, but simultaneously, should also try to arrange to cover all cases.
Regards, Martin _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
If you want signals to actually be handled in a timely manner, its best to leave the main thread of the program alone as a signal handling thread that just spends its time in a loop of time.sleep(123) calls rather than blocking on any sort of lock. Spawn other threads to do the actual work in your program. Signals are delivered indirectly in the existing CPython implementation by setting an internal flag that the main interpreter thread polls on occasion so blocking calls that do not interrupt and return early being called from the main thread will effectively block signals.
On Thu, 2 Jul 2009 15:47:48 -0700, "Gregory P. Smith" <greg@krypto.org> wrote:
On Mon, Jun 29, 2009 at 2:28 PM, "Martin v. Löwis"<martin@v.loewis.de> wrote:
AFAIK, ignoring EINTR doesn't preclude the calling of signal handlers.
This is my understanding as well - so I don't think Python actually "swallows" the signal.
A great example is reading from a socket. Whether or not it can be interrupted depends on the platform, so catching Ctrl+C often requires a timeout loop.
Also, remember that signals are asynchronous in the sense that they are handled outside the normal execution flow of a program. Checking for EINTR probably isn't the best way to determine if a signal has been sent to the program.
I think it would be reasonable to support "asynchronous" exceptions, and Python supports SIGINT fairly well most of the time.
It might be possible to support keyboard interrupts throughout the system, but changing Python to do so could also cause incompatibilities. So any change must be done with greatest care, but simultaneously, should also try to arrange to cover all cases.
Regards, Martin _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
If you want signals to actually be handled in a timely manner, its best to leave the main thread of the program alone as a signal handling thread that just spends its time in a loop of time.sleep(123) calls rather than blocking on any sort of lock. Spawn other threads to do the actual work in your program. Signals are delivered indirectly in the existing CPython implementation by setting an internal flag that the main interpreter thread polls on occasion so blocking calls that do not interrupt and return early being called from the main thread will effectively block signals.
Yes, this is all true now. The question is why the implementation works that way, and whether it is desirable to keep it working that way. Considering *some* of the lock implementations make themselves not interruptable by threads while others don't bother, it seems like *some* change to the status quo is desirable. Jean-Paul
On Thu, Jul 2, 2009 at 4:04 PM, Jean-Paul Calderone<exarkun@divmod.com> wrote:
On Thu, 2 Jul 2009 15:47:48 -0700, "Gregory P. Smith" <greg@krypto.org>
If you want signals to actually be handled in a timely manner, its best to leave the main thread of the program alone as a signal handling thread that just spends its time in a loop of time.sleep(123) calls rather than blocking on any sort of lock. Spawn other threads to do the actual work in your program. Signals are delivered indirectly in the existing CPython implementation by setting an internal flag that the main interpreter thread polls on occasion so blocking calls that do not interrupt and return early being called from the main thread will effectively block signals.
Yes, this is all true now. The question is why the implementation works that way, and whether it is desirable to keep it working that way.
Considering *some* of the lock implementations make themselves not interruptable by threads while others don't bother, it seems like *some* change to the status quo is desirable.
I'm probably to blame for this, but IIRC when we did it, lock acquisitions were not interruptible by signals. However, that was before posix threads were even standardized. Nowadays I believe we use pthreads everywhere except on Windows. But I know nearly nothing about pthreads. If pthreads mutex acquisition is (or can easily be made) interruptible by signals I'd be all for adding signal handling to them, assuming it doesn't slow things down much. Also, if there are ways to do the "try to acquire a lock or wait until a given timeout" natively and while handling signals, that would be a great improvement. I'll probably regret saying this within an hour though, -- --Guido van Rossum (home page: http://www.python.org/~guido/)
participants (6)
-
"Martin v. Löwis" -
Florian Mayer -
Gregory P. Smith -
Guido van Rossum -
Jean-Paul Calderone -
Phillip Sitbon