[Python-Dev] Making asyncio more thread-friendly
Hi, Celelibi, Welcome to Python Ideas. Python Dev is more for discussions of implementations of proposed features, typically clearly on their way to an accepted pull request into master. Python-Ideas is a better place for a request for enhancement without an implementation patch, so I'm moving it here. You're also more likely to get discussion here. Celelibi writes:
Hello world,
I always knew asynchronous programming is awesome. And having a language support for it really help its adoption by limiting the existence of incompatible event frameworks which would be a pain to use together.
However, the asynchronousness is very contaminating.
Yes. I would expect that.
This makes it difficult to make threads and asyncio coexist in a single application
True. Do you know of a programming environment that makes it easy that we can study?
Unless I'm mistaken, there's no obvious way to have a function run a coroutine in a given event loop, no matter if the loop in running or not, no matter if it's running in the current context or not. There are at least 3 cases that I can see: 1) The loop isn't running -> run_until_complete is the solution. 2) The loop is running in another context -> we might use call_soon_threadsafe and wait for the result. 3) The loop is running in the current context but we're not in a coroutine -> there's currently not much we can do without some major code change.
What I'd like to propose is that run_until_complete handle all three cases.
I don't see how the three can be combined safely and generically. I can imagine approaches that might work in a specific application, but the general problem is going to be very hard. async programming is awesome, and may be far easier to get 100% right than threading in some applications, but it has its pitfall too, as Nathaniel Smith points out: https://vorpus.org/blog/some-thoughts-on-asynchronous-api-design-in-a-post-a...
Did I miss an obvious way to make the migration from threads to asyncio easier?
Depends on what your threads are doing. If I were in that situation, I'd probably try to decouple the thread code from the async code by having the threads feed one or more queues into the async code base, and vice versa.
It becomes harder and harder to not believe that python is purposfully trying to make this kind of migration more painful.
Again, it would help a lot if you had an example of known working APIs we could implement, and/or more detail about your code's architecture to suggest workarounds. Regards, Steve
On Sun, Jun 07, 2020 at 03:47:05PM +0900, Stephen J. Turnbull wrote:
Hi, Celelibi,
Welcome to Python Ideas.
Python Dev is more for discussions of implementations of proposed features, typically clearly on their way to an accepted pull request into master. Python-Ideas is a better place for a request for enhancement without an implementation patch, so I'm moving it here.
If it's just that, I can probably provide a patch for run_until_complete.
This makes it difficult to make threads and asyncio coexist in a single application
True. Do you know of a programming environment that makes it easy that we can study?
Unfortunately no. But that doesn't mean we can't try to design asyncio to be more thread-friendly.
Unless I'm mistaken, there's no obvious way to have a function run a coroutine in a given event loop, no matter if the loop in running or not, no matter if it's running in the current context or not. There are at least 3 cases that I can see: 1) The loop isn't running -> run_until_complete is the solution. 2) The loop is running in another context -> we might use call_soon_threadsafe and wait for the result. 3) The loop is running in the current context but we're not in a coroutine -> there's currently not much we can do without some major code change.
What I'd like to propose is that run_until_complete handle all three cases.
I don't see how the three can be combined safely and generically. I can imagine approaches that might work in a specific application, but the general problem is going to be very hard. async programming is awesome, and may be far easier to get 100% right than threading in some applications, but it has its pitfall too, as Nathaniel Smith points out: https://vorpus.org/blog/some-thoughts-on-asynchronous-api-design-in-a-post-a...
I'm not sure what's hard about combining the three cases. The distinction between the three cases can be made easily with loop.is_running() and comparing asyncio.get_event_loop() to self. Implementing the third case can be a bit tricky but should be just a matter of making sure the loop itself behave nicely with its nested and parent instances. With a few important notes: - Infinite recursion is easily possible but it can be mitigated by: - Giving transitively a higher priority to tasks the current one is awaiting on. (Likely non-trivial to implement.) - Emiting a warning when the recursion depth goes beyon a given threshold. - This case as a whole would also likely deserve a warning if it's called directly from a coroutine.
Did I miss an obvious way to make the migration from threads to asyncio easier?
Depends on what your threads are doing. If I were in that situation, I'd probably try to decouple the thread code from the async code by having the threads feed one or more queues into the async code base, and vice versa.
It becomes harder and harder to not believe that python is purposfully trying to make this kind of migration more painful.
Again, it would help a lot if you had an example of known working APIs we could implement, and/or more detail about your code's architecture to suggest workarounds.
That's the thing. I don't know all the threads nor all the machinery in the application. If I did, I could probably make stronger assumptions. The application connects to several servers and relay messages beetween them. There's at least one thread per connection and one that process commands. I'm working on switching out a thread-based client protocol lib for an asyncio-based lib instead. To make the switch mostly invisible, I run an event loop in its own thread. When the server-connection-thread wants to send a message, it calls the usual method, which I modified to use call_soon_threadsafe to run a coroutine in the event loop and wait for the result. When an async callback is triggered by the lib, I just run the same old code which calls the whole machinery I do not control. By chance, it never end up wanting to call a coroutine. Therefore no need for a recursive call to the event loop... apparently... But if it did, I'd end up calling call_soon_threadsafe from the thread that runs the event loop and wait for the result. Which would just lock up. And that's what bugs me the most. To work around that, I would probably have to create a thread for the whole callback so that the event loop can get control back immediately. (And possibly create more threads. With all the bad things this entails.) Another issue I found is that the client protocol lib class isn't always instanciated by the same thread, and its constructor indirectly instanciate asyncio objects (like Event). In the end, its start method will start the event loop in its own thread. With the deprecation of the loop argument, Event() finds the loop it should use by calling get_event_loop() in its constructor. I therefore had to add a call set_event_loop(), even though this event loop will never run in this thread. That's a pretty ugly hack. The deprecation of the loop arguments looks like an incentive to create the asyncio objects in the threads that use them. Which seems pretty crazy to me as a whole. And practically, it means that the thread creation couldn't be part of the class itself, thus incurring more complexity to the outside code. As you can see I already have the workarounds I need. They're just pretty hack-ish. I was also a bit lucky. And I think these could be made simpler and prettier by implementing the suggested changes to run_until_complete(). Best regards, Celelibi
Thanks for switching to python-ideas. On Mon, Jun 8, 2020 at 12:35 AM Celelibi <celelibi@gmail.com> wrote:
On Sun, Jun 07, 2020 at 03:47:05PM +0900, Stephen J. Turnbull wrote:
Hi, Celelibi,
Welcome to Python Ideas.
Python Dev is more for discussions of implementations of proposed features, typically clearly on their way to an accepted pull request into master. Python-Ideas is a better place for a request for enhancement without an implementation patch, so I'm moving it here.
If it's just that, I can probably provide a patch for run_until_complete.
It would be premature until there is more agreement that your proposal is sound. I don't want to quote your whole email, but I do have an important piece of advice. Asyncio was designed explicitly to disallow recursive event loops. You may think they are important based on what you've done with other event loops in other languages. But changing asyncio to allow recursive event loops would require a very serious reconsideration of its design. Based on your project description it looks like you should ask somewhere (not python-ideas) for advice on how to best structure your gradual migration within asyncio's current limitations, rather than trying to propose deep changes to the standard library. Sorry the news is not better, but you will be better off this way. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
Fortunately, Guido has jumped in. He's the authority on asyncio. I recommend you consider his advice very carefully. Here are a couple more commments that I had already written when I saw his post. Celelibi writes:
But that doesn't mean we can't try to design asyncio to be more thread-friendly.
True, lack of an example to emulate doesn't mean we can't try ... but the fact that asyncio and threads are based on completely different models of concurrency likely does, in combination with that lack. Strong suggestion that "here be dragons".
I'm working on switching out a thread-based client protocol lib for an asyncio-based lib instead.
Have you tried to do this with an alternative to asyncio such as curio (by David Beazley, IIRC) or trio (by Nathaniel Smith)? They provide simpler APIs, and Nathaniel claims that they also provide a concurrency model that's far easier to understand. They're still async def/await-based, but they have a property Nathaniel calls "respect for causality" that seems very useful in understanding programs.
But if it did, I'd end up calling call_soon_threadsafe from the thread that runs the event loop and wait for the result. Which would just lock up.
If I understand correctly, this is the kind of thing that "respect for causality" helps to identify and to some extent prevent.
Another issue I found is that the client protocol lib class isn't always instanciated by the same thread, and its constructor indirectly instanciate asyncio objects (like Event).
The relations among the threading library, the asyncio library, and the client protocol library in your software are quite unclear to me. But this sounds very bad for your "divide and conquer" strategy. Asyncio is fundamentally a single-process, single-threaded, single-event-loop approach to handling concurrency. If the asyncio part of your program can ignore what other threads are doing, you might be OK, but this kind of coupling means you can't, and will need to do things like:
In the end, its start method will start the event loop in its own thread. With the deprecation of the loop argument, Event() finds the loop it should use by calling get_event_loop() in its constructor. I therefore had to add a call set_event_loop(), even though this event loop will never run in this thread. That's a pretty ugly hack.
But it's intended to be a temporary one, no? This is the kind of thing you need to do when you have multiple incompatible models of computation running in the same system.
The deprecation of the loop arguments looks like an incentive to create the asyncio objects in the threads that use them.
It is. A major point (perhaps the whole point) of asyncio is to provide that "single-process, single-threaded, single-event-loop approach" which nevertheless is extremely fast and scalable for I/O-bound programs. If you can't get to the point where all of your interactions with the threaded code can be done in a single thread, asyncio is very unlikely to be a good model for your system.
Which seems pretty crazy to me as a whole.
You should think carefully about that feeling. If it's just because you're temporarily in concurrency model hell, you'll get over that once the port to asyncio is done. But if you really need multiple threads because some code exposes them and you can't port that code to asyncio, you need to consider whether asyncio can do what you need. Steve
I would like to clarify that asyncio *does* most certainly provide support for programs that use multiple event loops and interact with a number of different threads. So, asyncio can definitely be used outside of a purely "single-process, single-threaded, single-event-loop approach". However, a key point that should be followed is that __async objects__ created in a single event loop are intended to be used only in that event loop. If you try to go outside of the bounds of that, you'll likely find yourself in the territory of undefined behavior (which is a part of why we're deprecating the loop argument throughout the high-level API, to discourage that). The asyncio part of your program can use a thread to run a function and get the result without blocking the event loop (using loop.run_in_executor(None, func, *args) or the upcoming asyncio.to_thread() in 3.9), schedule callbacks from another thread with loop.call_soon_threadsafe(), or submit a coroutine to be run in a specific event loop with asyncio.run_coroutine_threadsafe() from a different thread. But it does require some thinking about the architecture of your program, and can't just be used to replace an existing threaded program without any considerations. Fundamentally, OS threads and coroutines are two entirely different models of concurrency; though, we do have interoperability tools in place and are actively working on making them easier to utilize. Also, with adequate arguments for specific real-world use cases, those interoperability tools can be expanded upon as needed. However, these should be separate from existing parts of asyncio rather than changing methods like loop.run_until_complete(), IMO. There's something to be said about having too much purpose crammed into a single API member -- it tends to make the behavior needlessly complex and hard to follow. On Tue, Jun 9, 2020 at 2:37 AM Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Fortunately, Guido has jumped in. He's the authority on asyncio. I recommend you consider his advice very carefully. Here are a couple more commments that I had already written when I saw his post.
Celelibi writes:
But that doesn't mean we can't try to design asyncio to be more thread-friendly.
True, lack of an example to emulate doesn't mean we can't try ... but the fact that asyncio and threads are based on completely different models of concurrency likely does, in combination with that lack. Strong suggestion that "here be dragons".
I'm working on switching out a thread-based client protocol lib for an asyncio-based lib instead.
Have you tried to do this with an alternative to asyncio such as curio (by David Beazley, IIRC) or trio (by Nathaniel Smith)? They provide simpler APIs, and Nathaniel claims that they also provide a concurrency model that's far easier to understand. They're still async def/await-based, but they have a property Nathaniel calls "respect for causality" that seems very useful in understanding programs.
But if it did, I'd end up calling call_soon_threadsafe from the thread that runs the event loop and wait for the result. Which would just lock up.
If I understand correctly, this is the kind of thing that "respect for causality" helps to identify and to some extent prevent.
Another issue I found is that the client protocol lib class isn't always instanciated by the same thread, and its constructor indirectly instanciate asyncio objects (like Event).
The relations among the threading library, the asyncio library, and the client protocol library in your software are quite unclear to me. But this sounds very bad for your "divide and conquer" strategy. Asyncio is fundamentally a single-process, single-threaded, single-event-loop approach to handling concurrency. If the asyncio part of your program can ignore what other threads are doing, you might be OK, but this kind of coupling means you can't, and will need to do things like:
In the end, its start method will start the event loop in its own thread. With the deprecation of the loop argument, Event() finds the loop it should use by calling get_event_loop() in its constructor. I therefore had to add a call set_event_loop(), even though this event loop will never run in this thread. That's a pretty ugly hack.
But it's intended to be a temporary one, no? This is the kind of thing you need to do when you have multiple incompatible models of computation running in the same system.
The deprecation of the loop arguments looks like an incentive to create the asyncio objects in the threads that use them.
It is. A major point (perhaps the whole point) of asyncio is to provide that "single-process, single-threaded, single-event-loop approach" which nevertheless is extremely fast and scalable for I/O-bound programs.
If you can't get to the point where all of your interactions with the threaded code can be done in a single thread, asyncio is very unlikely to be a good model for your system.
Which seems pretty crazy to me as a whole.
You should think carefully about that feeling. If it's just because you're temporarily in concurrency model hell, you'll get over that once the port to asyncio is done. But if you really need multiple threads because some code exposes them and you can't port that code to asyncio, you need to consider whether asyncio can do what you need.
Steve _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/CFSVWP... Code of Conduct: http://python.org/psf/codeofconduct/
Kyle Stanley writes:
Fundamentally, OS threads and coroutines are two entirely different models of concurrency; though, we do have interoperability tools in place and are actively working on making them easier to utilize. Also, with adequate arguments for specific real-world use cases, those interoperability tools can be expanded upon as needed.
Could you be more specific about these tools? They sound like they may be what Celelibi is looking for. Just a list of APIs (even just a few core items) would likely point them at the right parts of the doc. I'll try if you can't respond, but this is well outside my experience, and my ocean of round tuits is looking like the Sahara Sea lately. Steve
Could you be more specific about these tools? They sound like they may be what Celelibi is looking for. Just a list of APIs (even just a few core items) would likely point them at the right parts of the doc.
Sorry if it wasn't clear, but by "interoperability tools", I was referring to the API members mentioned in the second paragraph of my previous message: loop.run_in_executor() (or the higher-level upcoming asyncio.to_thread() in 3.9) loop.call_soon_threadsafe() asyncio.run_coroutine_threadsafe() I consider these to be interoperability tools, as they can be used to bridge code that was previously built using an OS thread model with an async/await coroutine model, without having to entirely rewrite it. E.g. if some IO-bound function "func" was previously passed to the *target* argument of threading.Thread(), you could pass it to loop.run_in_executor(None, func, *args) or the upcoming asyncio.to_thread(func, *args) instead. I'm using loop.run_in_executor() as an example because that's the one I've used the most in my own code. Of course, it would likely result in improved performance if it were to be entirely rewritten instead of still using a thread, but this isn't always possible. Especially in cases where the underlying OS call doesn't have async support. Also, it can require a significant amount of time investment when working with a large existing library, so using something like loop.run_in_executor() allows it to be used and tested with the rest of the async program while working on a larger rewrite. I believe we're also going to be working on an asyncio.from_thread() in the near future (3.10?), which would presumably be for executing a function in a *specific* thread/event loop. AFAIK, there are still some details to work out though, such as how the context will be specified. I'm not certain if it would make more sense to use a unique token (Trio does this), or use an event loop argument like we've done for other parts of asyncio. Personally, I'm somewhat inclined towards something like a token or other option since we're removing the event loop arg from most of the high-level API. The OPs issue highlights a difficult balance from an API design perspective. Having constant access to the event loop by using it to call/schedule everything and being able to pass it around as an argument makes it easier to access when working with multiple event loops (loops outside of the main thread, specifically), but it also adds a significant amount of additional boilerplate and room for error (such as attempting to share async objects between multiple event loops). While I think eventually removing the event loop arg is a good thing, we may need to consider delaying the removal until we're certain that users have access to a full array of tools for working with multiple event loops. It was deprecated in 3.8 and scheduled for removal in 3.10, but that might be a bit too soon (even more so with the release cadence being shortened recently). On Tue, Jun 9, 2020 at 11:09 PM Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Kyle Stanley writes:
Fundamentally, OS threads and coroutines are two entirely different models of concurrency; though, we do have interoperability tools in place and are actively working on making them easier to utilize. Also, with adequate arguments for specific real-world use cases, those interoperability tools can be expanded upon as needed.
Could you be more specific about these tools? They sound like they may be what Celelibi is looking for. Just a list of APIs (even just a few core items) would likely point them at the right parts of the doc. I'll try if you can't respond, but this is well outside my experience, and my ocean of round tuits is looking like the Sahara Sea lately.
Steve
participants (4)
-
Celelibi -
Guido van Rossum -
Kyle Stanley -
Stephen J. Turnbull