On Tue, Feb 19, 2019 at 7:41 PM Nathaniel Smith <njs@pobox.com> wrote:
I'm not sure what a "higher priority" exception is...raising an exception is hard to miss.
Just quickly on this one point: that was just my colloquial way of saying superclass (or at least not a subclass), to emphasize that if you're e.g. catching CancelledError, this new exception would bubble up whereas CancelledError wouldn't. A similar example is KeyboardInterrupt not being caught by Exception. In the case of CancelledError, we probably would want the two exceptions to be comparable to one another rather than incomparable. I seem to recall for example there being an old discussion as to whether CancelledError should inherit from Exception or not. --Chris
There are a few things Trio does differently here that might be relevant, but it depends on why Chris is having trouble cancelling tasks.
1. Trio's cancellation exception, trio.Cancelled, inherits from BaseException instead of Exception, like KeyboardInterrupt or StopIteration. So 'except Exception' doesn't catch it by accident.
2. Trio's cancellation is "stateful": if your code is in the cancelled state, then every time you try to do an async operation then it raises trio.Cancelled again. So you avoid the case where a tasks gets stuck, someone forces it to raise CancelledError, and then it has a 'finally' block that tries to do some cleanup... but the 'finally' block also gets stuck. In trio the 'finally' block can't accidentally get stuck.
3. Both of these features are somewhat dependent on trio using "delimited" cancellation. Before you can cancel something, you have to say how far you want to unwind. This means that there's never any reason for anyone to try to catch 'Cancelled' on purpose, because trio will catch it for you at the appropriate moment. And it's hard to do 'stateful' cancellation if you don't know how long the state is supposed to persist. And, you avoid cases where some code thinks it just threw in a CancelledError and is supposed to catch it, but actually it was thrown in from some other stack frame, and it ends up confusedly catching the wrong exception.
I'm not sure how much of this could be adapted for asyncio. The obvious change would be to make asyncio.CancelledError a BaseException, though it seems borderline to me from a back-compat perspective. I think I remember Yury was thinking about changing it anyway, though? That would definitely help with the 'except Exception' kind of mistake.
But the other issues are deeper. If you don't have a solid system for keeping track of what exactly is supposed to be cancelled, then it's easy to accidentally cancel too much, or cancel too little. Solving that requires a systematic approach. And unfortunately, asyncio already has 2 different sets of cancellation semantics (Future.cancel -> takes effect immediately, irrevocable & idempotent, doesn't necessarily cause the underlying machinery to stop processing, just stops it from reporting its result; Task.cancel -> doesn't take effect immediately or necessarily at all, can be called repeatedly and injects one CancelledError per call, tries to stop the underlying machinery, chains to other tasks/futures that the first task is await'ing). So if our goal is to make the system as a whole as reliable and predictable as possible within the constraints of back-compat... I don't know whether adding a third set of semantics would actually help, or make more code confused about what it was supposed to be catching.
And I don't know if any of these actually address whatever problem you're having with uncancellable tasks. It's certainly possible to make an uncancellable task in Trio too. We just try to make it hard to do by accident.
-n
On Tue, Feb 19, 2019 at 3:53 PM Chris Jerdonek <chris.jerdonek@gmail.com> wrote:
On Tue, Feb 19, 2019 at 12:33 PM Yury Selivanov <yselivanov@gmail.com>
Unfortunately asyncio isn't super flexible around "cancellation with a
timeout" kind of scenarios. The current assumption is that once the cancellation is requested, the Task will start cancelling and will do so in a timely manner. Imposing a second layer of timeouts on the cancellation
wrote: process itself isn't natively supported. But to properly address this we don't need a very broadly defined Task.set_exception(); we need to rethink the cancellation in asyncio (perhaps draw some inspiration from Trio and other frameworks).
What options have you already considered for asyncio's API? A couple
naive things that occur to me are adding task.kill() with higher priority than task.cancel() (or equivalently task.graceful_cancel() with lower priority). Was task.cancel() meant more to have the meaning of "kill" or "graceful cancel"? In addition, the graceful version of the two (whichever that may be) could accept a timeout argument -- after which the exception of higher priority is raised. I realize this is a more simplistic model compared to the options trio is considering, but asyncio has already gone down the path of the simpler approach.
--Chris
Yury
_______________________________________________ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
-- Nathaniel J. Smith -- https://vorpus.org