On 5 October 2016 at 02:15, Guido van Rossum
Honestly it feels like many things can go wrong with this API model, esp. you haven't answered what should happen when a method of SomeClass (either a synchronous one or an async one) calls run_in_foreground() on something -- or, more likely, calls some harmless-looking function that calls another harmless-looking function that calls run_in_foreground(). At that point you have pre-emptive scheduling back in play (or your coroutines may be blocked unnecessarily) and I think you have nothing except a more complicated API to work with threads.
Yeah, that's the main reason I haven't gone beyond this as a toy idea - there are so many ways to get yourself in trouble if you don't already understand the internal details.
I think I am ready to offer a counterproposal where the event loop runs in one thread and synchronous code runs in another thread and we give the synchronous code a way to synchronously await a coroutine or an asyncio.Future. This can be based on asyncio.run_coroutine_threadsafe(), which takes a coroutine or an asyncio.Future and returns a concurrent Future. (It also takes a loop, and it assumes that loop runs in a different thread. I think it should assert that.)
Oh, that makes a lot more sense, as we'd end up with a situation where async code gets used in one of two ways: - asynchronous main thread (the typical way it gets used now) - synchronous thread with a linked asynchronous helper thread The key differences between the latter and a traditional thread pool is that there'd only be the *one* helper thread for any given synchronous thread, and as long as the parent thread keeps its hands off any shared data structures while coroutines are running, you can still rely on async/await to interleave access to data structures shared by the coroutines.
The main feature of my counterproposal as I see it is that async code should not call back into synchronous code, IOW once you are writing coroutines, you have to use the coroutine API for everything you do. And if something doesn't have a coroutine API, you run it in a background thread using loop.run_in_executor().
So either you buy into the async way of living and it's coroutines all the way down from there, no looking back -- or you stay on the safe side of the fence, and you interact with coroutines only using a very limited "remote manipulator" API. The two don't mix any better than that.
+1 I considered suggesting that the "remote manipulator" API could be spelled "await expr", but after starting to write that idea up, realised it was likely a recipe for hard-to-debug problems when folks forget to add the "async" declaration to a coroutine definition. So that would instead suggest 2 module level functions in asyncio: * call_in_background(coroutine_or_callable, *args, **kwds): - creates the helper thread if it doesn't already exist, stores a reference in a thread local variable - schedules coroutines directly in the helper thread's event loop - schedules other callables in the helper thread's executor - returns an asyncio.Future instance - perhaps lets the EventLoopPolicy override this default behaviour? * wait_for_result: - blocking call that waits for asyncio.Future.result() to be ready Using "call_in_background" from a coroutine would be OK, but somewhat redundant (as if a coroutine is already running, you could just use the current thread's event loop instead). Using "wait_for_result" from a coroutine would be inappropriate, as with any other blocking call. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia