On Wed, Jun 28, 2017 at 3:19 PM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Erik Bray wrote:
At this point a potentially waiting SIGINT is handled, resulting in KeyboardInterrupt being raised while inside the with statement's suite, and finally block, and hence Lock.__exit__ are entered.
Seems to me this is the behaviour you *want* in this case, otherwise the lock can be acquired and never released. It's disconcerting that it seems to be very difficult to get that behaviour with a pure Python implementation.
I think normally you're right--this is the behavior you would *want*, but not the behavior that's consistent with how Python implements the `with` statement, all else being equal. Though it's still not entirely fair either because if Lock.__enter__ were pure Python somehow, it's possible the exception would be raised either before or after the lock is actually marked as "acquired", whereas in the C implementation acquisition of the lock will always succeed (assuming the lock was free, and no other exceptional conditions) before the signal handler is executed.
I think it might be possible to gain more consistency between these cases if pending signals are checked/handled after any direct call to PyCFunction from within the ceval loop.
IMO that would be going in the wrong direction by making the C case just as broken as the Python case.
Instead, I would ask what needs to be done to make this work correctly in the Python case as well as the C case.
You have a point there, but at the same time the Python case, while "broken" insofar as it can lead to broken code, seems correct from the Pythonic perspective. The other possibility would be to actually change the semantics of the `with` statement. Or as you mention below, a way to temporarily mask signals...
I don't think it's even possible to write Python code that does this correctly at the moment. What's needed is a way to temporarily mask delivery of asynchronous exceptions for a region of code, but unless I've missed something, no such facility is currently provided.
What would such a facility look like? One possibility would be to model it on the sigsetmask() system call, so there would be a function such as
mask_async_signals(bool)
that turns delivery of async signals on or off.
However, I don't think that would work. To fix the locking case, what we need to do is mask async signals during the locking operation, and only unmask them once the lock has been acquired. We might write a context manager with an __enter__ method like this:
def __enter__(self): mask_async_signals(True) try: self.acquire() finally: mask_async_signals(False)
But then we have the same problem again -- if a Keyboard Interrupt occurs after mask_async_signals(False) but before __enter__ returns, the lock won't get released.
Exactly.
Another approach would be to provide a context manager such as
async_signals_masked(bool)
Then the whole locking operation could be written as
with async_signals_masked(True): lock.acquire() try: with async_signals_masked(False): # do stuff here finally: lock.release()
Now there's no possibility for a KeyboardInterrupt to be delivered until we're safely inside the body, but we've lost the ability to capture the pattern in the form of a context manager.
The only way out of this I can think of at the moment is to make the above pattern part of the context manager protocol itself. In other words, async exceptions are always masked while the __enter__ and __exit__ methods are executing, and unmasked while the body is executing.
I think so too. That's more or less in line with Nick's idea on njs's issue (https://bugs.python.org/issue29988) of an ATOMIC_UNTIL opcode. That's just one implementation possibility. My question would be to make that a language-level requirement of the context manager protocol, or just something CPython does... Thanks, Erik