[New-bugs-announce] [issue26357] asyncio.wait loses coroutine return value

[André Caron]

New submission from André Caron:

When the asyncio.wait() function with coroutine objects as inputs, it is impossbible to extract the results reliably.

This issue arises because asyncio.wait() returns an unordered set of futures against which membership tests of coroutine objects always fail.

The issue is made even worse by the fact that coroutine objects cannot be awaited multiple times (see https://bugs.python.org/issue25887).  Any await expression on the coroutine object after the call to asyncio.wait() returns None, regardless of the coroutine's return value.

  (See attached `asyncio-wait-coroutine.py` for an example of both these issues.)

In the worst case, multiple inputs are coroutine objects and the set of returned futures contains return values but there is no way to determine which result corresponds to which coroutine call.

To work around this issue, callers need to explicitly use asyncio.ensure_future() on coroutine objects before calling asyncio.wait().

  (See comment in `asyncio-wait-coroutine.py` for an example "fix").

Note that, in general, it is not possible to know ahead of time whether all inputs to asyncio.wait() are coroutines or futures.  Furthermore, the fact that a given third-party library function is implemented as a regular function that returns a Future or a proper coroutine is an implementation decision which may not be part of the public interface.  Even if it is, the inputs to asyncio.wait() may come from complex code paths and it may be difficult to verify that all of them end up producing a Future.

As a consequence, the only reliable way to recover all results from asyncio.wait() is to explicitly call asyncio.ensure_future() on each of the inputs.

When doing so, both the membership test against the `done` set and the await expressions work as expected.

Quickly, there are several possible solutions:
- allow programs to await coroutine multiple times;
- make the set membership test of a coroutine object succeed; or
- change support for coroutine objects as inputs to asyncio.wait():
** update documentation for asyncio.wait() to explain this limitation; or
** explicitly reject coroutine objects; or
** warn when passing coroutine objects as inputs -- unless wrapped.

Related issue: https://bugs.python.org/issue25887 proposes a patch to change the behaviour of awaiting a coroutine object multiple times in order to produce a RuntimeError on all awaits after the 1st.  While that change in behaviour would make it easier to diagnose the loss of the return value, it does not fix this issue.

----------
components: asyncio
files: asyncio-wait-coroutine.py
messages: 260250
nosy: André Caron, gvanrossum, haypo, yselivanov
priority: normal
severity: normal
status: open
title: asyncio.wait loses coroutine return value
versions: Python 3.5
Added file: http://bugs.python.org/file41914/asyncio-wait-coroutine.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26357>
_______________________________________