Could you be more specific about these tools? They sound like they
may be what Celelibi is looking for. Just a list of APIs (even just a few core items) would likely point them at the right parts of the doc.
Sorry if it wasn't clear, but by "interoperability tools", I was referring to the API members mentioned in the second paragraph of my previous message:
loop.run_in_executor() (or the higher-level upcoming asyncio.to_thread() in 3.9) loop.call_soon_threadsafe() asyncio.run_coroutine_threadsafe()
I consider these to be interoperability tools, as they can be used to bridge code that was previously built using an OS thread model with an async/await coroutine model, without having to entirely rewrite it. E.g. if some IO-bound function "func" was previously passed to the *target* argument of threading.Thread(), you could pass it to loop.run_in_executor(None, func, *args) or the upcoming asyncio.to_thread(func, *args) instead. I'm using loop.run_in_executor() as an example because that's the one I've used the most in my own code.
Of course, it would likely result in improved performance if it were to be entirely rewritten instead of still using a thread, but this isn't always possible. Especially in cases where the underlying OS call doesn't have async support. Also, it can require a significant amount of time investment when working with a large existing library, so using something like loop.run_in_executor() allows it to be used and tested with the rest of the async program while working on a larger rewrite.
I believe we're also going to be working on an asyncio.from_thread() in the near future (3.10?), which would presumably be for executing a function in a *specific* thread/event loop. AFAIK, there are still some details to work out though, such as how the context will be specified. I'm not certain if it would make more sense to use a unique token (Trio does this), or use an event loop argument like we've done for other parts of asyncio. Personally, I'm somewhat inclined towards something like a token or other option since we're removing the event loop arg from most of the high-level API.
The OPs issue highlights a difficult balance from an API design perspective. Having constant access to the event loop by using it to call/schedule everything and being able to pass it around as an argument makes it easier to access when working with multiple event loops (loops outside of the main thread, specifically), but it also adds a significant amount of additional boilerplate and room for error (such as attempting to share async objects between multiple event loops).
While I think eventually removing the event loop arg is a good thing, we may need to consider delaying the removal until we're certain that users have access to a full array of tools for working with multiple event loops. It was deprecated in 3.8 and scheduled for removal in 3.10, but that might be a bit too soon (even more so with the release cadence being shortened recently).
On Tue, Jun 9, 2020 at 11:09 PM Stephen J. Turnbull < firstname.lastname@example.org> wrote:
Kyle Stanley writes:
Fundamentally, OS threads and coroutines are two entirely different models of concurrency; though, we do have interoperability tools in place and are actively working on making them easier to utilize. Also, with adequate arguments for specific real-world use cases, those interoperability tools can be expanded upon as needed.
Could you be more specific about these tools? They sound like they may be what Celelibi is looking for. Just a list of APIs (even just a few core items) would likely point them at the right parts of the doc. I'll try if you can't respond, but this is well outside my experience, and my ocean of round tuits is looking like the Sahara Sea lately.