correct way to combine two coroutines in a loop

Hey everyone, I hope this is the right mlist to post my question to. If not, please redirect me to the correct one. I recently tried async for websockets. Here's the question: Looking at the following code, is there a better way to do it? I personally find it much too much code for its purpose. Yet I cannot think of an improvement because I am not used to the usual patterns in async programming. What I find especially annoying is the boilerplate code regarding the renewal of the consumed task object (initializing with None, picking the last one or create a new one, resetting to None, repeat). If, due to the boilerplate, the intent is not clear, there are two endlessly producing coroutines ("queue" & "receive") from which is being read and reacted to, whatever comes first. notify_task = None event_task = None while True: notify_task = notify_task or asyncio.Task(queue.get()) event_task = event_task or asyncio.Task(receive()) done, pending = await asyncio.wait([notify_task, event_task], return_when=asyncio.FIRST_COMPLETED) if notify_task in done: notify = notify_task.result() notify_task = None await handle_notify(scope, send, notify) if event_task in done: event = event_task.result() event_task = None if await handle_ws_default_protocol(scope, send, event, conn, listen_callback): break Why am I asking? Because a friend of mine also was a bit puzzled that this "select.select"-type of programming is so verbose in async or if we were just missing a crucial element. Cheers and thanks you in advance for any answer, Sven PS: What's the history of this code? Reading different tutorials, both queues are discussed and implemented separately (one is websocket, the other one is a message broker). Either code examples were quiet small. But as soon as you want to plug both together, the resulting code suddenly explodes and requires a lot of manual handling as shown above. At least from my perception of "manual". I do not blame these tutorials as they focus on either problem separately. Still in real-world projects, different concepts frequently need to be combined to make the whole idea work. It took me quite some time to find a way to do it. The result above, I find it neither readable nor explainable to other async-noobs like me. Especially "asyncio.wait" was quite hidden in the library (and from google) and it took me many iterations to understand how to interact with it properly (although the obvious API is quite simple).

Why not have two independent coroutines that each loop forever, handling events from their respective queues? On Tue, Jul 13, 2021 at 12:04 PM Sven R. Kunze <srkunze@mail.de> wrote:
Hey everyone,
I hope this is the right mlist to post my question to. If not, please redirect me to the correct one. I recently tried async for websockets. Here's the question:
Looking at the following code, is there a better way to do it?
I personally find it much too much code for its purpose. Yet I cannot think of an improvement because I am not used to the usual patterns in async programming. What I find especially annoying is the boilerplate code regarding the renewal of the consumed task object (initializing with None, picking the last one or create a new one, resetting to None, repeat).
If, due to the boilerplate, the intent is not clear, there are two endlessly producing coroutines ("queue" & "receive") from which is being read and reacted to, whatever comes first.
notify_task = None event_task = None while True: notify_task = notify_task or asyncio.Task(queue.get()) event_task = event_task or asyncio.Task(receive()) done, pending = await asyncio.wait([notify_task, event_task], return_when=asyncio.FIRST_COMPLETED)
if notify_task in done: notify = notify_task.result() notify_task = None await handle_notify(scope, send, notify)
if event_task in done: event = event_task.result() event_task = None if await handle_ws_default_protocol(scope, send, event, conn, listen_callback): break
Why am I asking?
Because a friend of mine also was a bit puzzled that this "select.select"-type of programming is so verbose in async or if we were just missing a crucial element.
Cheers and thanks you in advance for any answer, Sven
PS:
What's the history of this code?
Reading different tutorials, both queues are discussed and implemented separately (one is websocket, the other one is a message broker). Either code examples were quiet small. But as soon as you want to plug both together, the resulting code suddenly explodes and requires a lot of manual handling as shown above. At least from my perception of "manual".
I do not blame these tutorials as they focus on either problem separately. Still in real-world projects, different concepts frequently need to be combined to make the whole idea work.
It took me quite some time to find a way to do it. The result above, I find it neither readable nor explainable to other async-noobs like me. Especially "asyncio.wait" was quite hidden in the library (and from google) and it took me many iterations to understand how to interact with it properly (although the obvious API is quite simple).
_______________________________________________ Async-sig mailing list -- async-sig@python.org To unsubscribe send an email to async-sig-leave@python.org https://mail.python.org/mailman3/lists/async-sig.python.org/ Code of Conduct: https://www.python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>

There's no interdependency in your code snippet. If that's really the case, then why not refactor it into something like: async def notify_all(): async for x in queue: await handle_notification(x) async def receive_all(): async for x in receive: await handle_reception(x) await asyncio.gather(notify_all(), receive_all()) On Wed, 14 Jul 2021 at 04:04, Sven R. Kunze <srkunze@mail.de> wrote:
Hey everyone,
I hope this is the right mlist to post my question to. If not, please redirect me to the correct one. I recently tried async for websockets. Here's the question:
Looking at the following code, is there a better way to do it?
I personally find it much too much code for its purpose. Yet I cannot think of an improvement because I am not used to the usual patterns in async programming. What I find especially annoying is the boilerplate code regarding the renewal of the consumed task object (initializing with None, picking the last one or create a new one, resetting to None, repeat).
If, due to the boilerplate, the intent is not clear, there are two endlessly producing coroutines ("queue" & "receive") from which is being read and reacted to, whatever comes first.
notify_task = None event_task = None while True: notify_task = notify_task or asyncio.Task(queue.get()) event_task = event_task or asyncio.Task(receive()) done, pending = await asyncio.wait([notify_task, event_task], return_when=asyncio.FIRST_COMPLETED)
if notify_task in done: notify = notify_task.result() notify_task = None await handle_notify(scope, send, notify)
if event_task in done: event = event_task.result() event_task = None if await handle_ws_default_protocol(scope, send, event, conn, listen_callback): break
Why am I asking?
Because a friend of mine also was a bit puzzled that this "select.select"-type of programming is so verbose in async or if we were just missing a crucial element.
Cheers and thanks you in advance for any answer, Sven
PS:
What's the history of this code?
Reading different tutorials, both queues are discussed and implemented separately (one is websocket, the other one is a message broker). Either code examples were quiet small. But as soon as you want to plug both together, the resulting code suddenly explodes and requires a lot of manual handling as shown above. At least from my perception of "manual".
I do not blame these tutorials as they focus on either problem separately. Still in real-world projects, different concepts frequently need to be combined to make the whole idea work.
It took me quite some time to find a way to do it. The result above, I find it neither readable nor explainable to other async-noobs like me. Especially "asyncio.wait" was quite hidden in the library (and from google) and it took me many iterations to understand how to interact with it properly (although the obvious API is quite simple).
_______________________________________________ Async-sig mailing list -- async-sig@python.org To unsubscribe send an email to async-sig-leave@python.org https://mail.python.org/mailman3/lists/async-sig.python.org/ Code of Conduct: https://www.python.org/psf/codeofconduct/
participants (3)
-
Dima Tisnek
-
Guido van Rossum
-
Sven R. Kunze