[issue37736] asyncio.wait_for is still confusing

[David Lewis]

New submission from David Lewis <dlewis at vivcourt.com>:

This issue is a follow up to previous discussions about confusing results with asyncio.wait_for. In the current implementation, it seems unintuitive that a coroutine with a timeout argument may easily wait forever. Perhaps wait_for could use an await_cancellation=True kwarg.

Prior issues:

a) "It's a feature, not a bug" - Guido
https://github.com/python/asyncio/issues/253#issuecomment-120020018

b) "I don't feel comfortable with asyncio living with this bug till 3.8." - Yury
https://bugs.python.org/issue32751#msg318065

Originally, wait_for would cancel the future and raise TimeoutError immediately. In the case of a Task, it could remain active for some time after the timeout, since its cancellation is potentially asynchronous.

In (a), this behaviour was defended, since waiting on the cancellation would violate the implicit contract of the timeout argument to wait_for(). While the documentation suggests it's a poor idea, it's not illegal for a task to defer or entirely refuse cancellation.

In (b), the task outliving the TimeoutError was considered a bug, and the behaviour changed to its current state. To address the issue raised in (a), the documentation for wait_for now contains the line "The function will wait until the future is actually cancelled, so the total wait time may exceed the timeout."

However, we still have the case where a misbehaving Task can cause wait_for to hang indefinitely. For example, the following program doesn't terminate:

import asyncio, contextlib

async def bad():
    while True:
        with contextlib.suppress(asyncio.CancelledError):
            await asyncio.sleep(1)
            print("running...")

if __name__ == '__main__':
    asyncio.run(asyncio.wait_for(bad(), 1))


More realistically, the task may have cooperative cancellation logic that waits for something else to be tidied up:

try:
    await wait_for(some_task(service), 10)
except TimeoutError:
   ...
finally:
    service.stop()

----------
components: asyncio
messages: 348848
nosy: David Lewis, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: asyncio.wait_for is still confusing
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue37736>
_______________________________________