[New-bugs-announce] [issue46752] Introduce task groups to asyncio and change task cancellation semantics

[Guido van Rossum]

New submission from Guido van Rossum <guido at python.org>:

After some conversations with Yury, and encouraged by the SC's approval of PEP 654, I am proposing to add a new class, asyncio.TaskGroup, which introduces structured concurrency similar to nurseries in Trio.

I started with EdgeDb's TaskGroup implementation (https://github.com/edgedb/edgedb/blob/master/edb/common/taskgroup.py) and tweaked it only slightly. I also changed a few things in asyncio.Task (see below).

The key change I made to EdgeDb's TaskGroup is that subtasks can keep spawning more subtasks while __aexit__ is running; __aexit__ exits once the last subtask is done. I made this change after consulting some Trio folks, who knew of real-world use cases for this behavior, and did not know of real-world code in need of prohibiting task creation as soon as __aexit__ starts running. I added some tests for the new behavior; none of the existing tests needed to be adjusted to accommodate this change.

(For other changes relative to the EdgeDb's TaskGroup, see GH-31270.)

In order to avoid the need to monkey-patch the parent task, I added two new methods to asyncio.Task, .cancelled() and .uncancel(), that manage a flag corresponding to __cancel_requested__ in EdgeDb's TaskGroup. 

**This introduces a change in behavior around task cancellation:**

* A task that catches CancelledError is allowed to run undisturbed (ignoring further .cancel() calls and allowing any number of await calls!) until it either exits or calls .uncancel().

This change in semantics did not cause any asyncio unittests to fail. However, it may be surprising (especially to Trio folks, where the semantics are pretty much the opposite, once a Trio task is cancelled all further await calls in that task fail unless explicitly shielded).

For the TaskGroup tests to pass, we require a flag that is not cleared. However, it is probably not really required to ignore subsequent .cancel() calls until .uncancel() is called. This just seemed more consistent, and it is what @asvetlov proposed above and implemented in GH-31313 (using a property .__cancel_requested__ as the API).

----------
assignee: gvanrossum
components: asyncio
keywords: needs review
messages: 413260
nosy: asvetlov, gvanrossum, iritkatriel, yselivanov
priority: normal
severity: normal
stage: patch review
status: open
title: Introduce task groups to asyncio and change task cancellation semantics
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue46752>
_______________________________________