asyncio: return from multiple coroutines
Hey everyone. I have been looking into asyncio lately, and even though I have had my fair share of work, I still have some of it very shaky, so first of all forgive me if what I am saying here is already implemented and I totally missed it (so far, it looks *really* likely). Basically this is the situation: I have an application that listens on two websockets through the async library https://websockets.readthedocs.io/ and I have to perform the same function on the result, no matter where the message came from. I understand that I can create two coroutines that call the same function, but it would be much cleaner (because of implementation issues) if I can simply create a coroutine that yields the result of whichever connection arrives first. I have implemented a rather cumbersome solution with async Queues (as I would do in threading), in which each connection puts its message in a queue and an adapter class awaits the first element of the queue on "receive". Here I attach a pastebin with the minimal working example: https://pastebin.com/BzaxRbtF However, it looks like a more async-friendly solution should exist, something like ``` async def _start(): msg1 = recv("Messager 1", sleep_time=1) msg2 = recv("Messager 2", sleep_time=2) while True: result = await asyncio.on_first_return(msg1, msg2) print(result) ``` (I understand that this implementation would not work because the event loop doesn't know that it is "the same task repeated", but it's just to tell you the general idea) Again, it's quite likely I am not seeing something obvious, but I didn't know where else to ask. Thank you very much, Pablo
I believe asyncio.wait() with "return_when=FIRST_COMPLETED" would perform the functionality you're looking for with the "asyncio.on_first_return()". For details on the functionality of asyncio.wait(), see https://docs.python.org/3/library/asyncio-task.html#asyncio.wait.
I understand that I can create two coroutines that call the same function, but it would be much cleaner (because of implementation issues) if I can simply create a coroutine that yields the result of whichever connection arrives first.
You can use an asynchronous generator that will continuously yield the result of the first recv() that finishes (I'm assuming you mean "yields" literally and want multiple results from a generator, but I might be misinterpreting that part). Here's a brief example, using the recv() coroutine function from the pastebin linked: ``` import asyncio import random async def recv(message: str, max_sleep: int): sleep_time = max_sleep * random.random() await asyncio.sleep(sleep_time) return f'{message} awaited for {sleep_time:.2f}s' async def _start(): while True: msgs = [ asyncio.create_task(recv("Messager 1", max_sleep=1)), asyncio.create_task(recv("Messager 2", max_sleep=1)) ] done, _ = await asyncio.wait(msgs, return_when=asyncio.FIRST_COMPLETED) result = done.pop() yield await result async def main(): async for result in _start(): print(result) asyncio.run(main()) ``` Note that in the above example, in "msgs", you can technically pass the coroutine objects directly to asyncio.wait(), as they will be implicitly converted to tasks. However, we decided to deprecate that functionality in Python 3.8 since it can be rather confusing. So creating and passing the tasks is a better practice.
Again, it's quite likely I am not seeing something obvious, but I didn't know where else to ask.
If you're not mostly certain or relatively inexperienced with the specific area that the question pertains to, I'd recommend asking on python-list first (or another Python user community). python-ideas is primarily intended for new feature proposals/suggestions. Although if you've tried other resources and haven't found an answer, it's perfectly fine to ask a question as part of the suggestion post. On Mon, Jun 22, 2020 at 6:24 PM Pablo Alcain <pabloalcain@gmail.com> wrote:
I believe asyncio.wait() with "return_when=FIRST_COMPLETED" would perform the functionality you're looking for with the "asyncio.on_first_return()". For details on the functionality of asyncio.wait(), see https://docs.python.org/3/library/asyncio-task.html#asyncio.wait.
I understand that I can create two coroutines that call the same function, but it would be much cleaner (because of implementation issues) if I can simply create a coroutine that yields the result of whichever connection arrives first.
You can use an asynchronous generator that will continuously yield the result of the first recv() that finishes (I'm assuming you mean "yields" literally and want multiple results from a generator, but I might be misinterpreting that part). Here's a brief example, using the recv() coroutine function from the pastebin linked: ``` import asyncio import random async def recv(message: str, max_sleep: int): sleep_time = max_sleep * random.random() await asyncio.sleep(sleep_time) return f'{message} awaited for {sleep_time:.2f}s' async def _start(): while True: msgs = [ asyncio.create_task(recv("Messager 1", max_sleep=1)), asyncio.create_task(recv("Messager 2", max_sleep=1)) ] done, _ = await asyncio.wait(msgs, return_when=asyncio.FIRST_COMPLETED) result = done.pop() yield await result async def main(): async for result in _start(): print(result) asyncio.run(main()) ``` Note that in the above example, in "msgs", you can technically pass the coroutine objects directly to asyncio.wait(), as they will be implicitly converted to tasks. However, we decided to deprecate that functionality in Python 3.8 since it can be rather confusing. So creating and passing the tasks is a better practice.
Again, it's quite likely I am not seeing something obvious, but I didn't know where else to ask.
If you're not mostly certain or relatively inexperienced with the specific area that the question pertains to, I'd recommend asking on python-list first (or another Python user community). python-ideas is primarily intended for new feature proposals/suggestions. Although if you've tried other resources and haven't found an answer, it's perfectly fine to ask a question as part of the suggestion post. On Mon, Jun 22, 2020 at 6:24 PM Pablo Alcain <pabloalcain@gmail.com> wrote:
participants (2)
-
Kyle Stanley -
Pablo Alcain