[Python-ideas] async objects

[Guido van Rossum]

On Tue, Oct 4, 2016 at 4:30 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>     class SomeClass(object):
>         def some_sync_method(self):
>             return 42
>         async def some_async_method(self):
>             await asyncio.sleep(3)
>             return 42
>
> o = auto_schedule(SomeClass())  # Indicating that the user wants an
> async version of the object
> r1 = o.some_sync_method() # Automatically run in a background thread
> r2 = o.some_async_method() # Automatically scheduled as a coroutine
> print(run_in_foreground(r1))
> print(run_in_foreground(r2))

So maybe r1 and r2 are just concurrent.futures.Futures, and
run_in_foreground(r) wraps r.result(). And auto_schedule() is a proxy
that turns all method calls into async calls with a (concurrent)
Future to wait for the result. There's an event loop somewhere that
sits idle except when you call run_in_foreground() on somethong; it's
only used for the async methods, since the sync methods run in a
background thread (pool, I hope). Or perhaps r2 is an asyncio.Future
and run_in_foreground(r2) wraps loop.run_until_complete(r2). I suppose
the event loop should also be activated when waiting for r1, so maybe
r1 should be an asyncio Future that wraps a concurrent Future (using
asyncio.wrap_future(), which can do just that thing).

Honestly it feels like many things can go wrong with this API model,
esp. you haven't answered what should happen when a method of
SomeClass (either a synchronous one or an async one) calls
run_in_foreground() on something -- or, more likely, calls some
harmless-looking function that calls another harmless-looking function
that calls run_in_foreground(). At that point you have pre-emptive
scheduling back in play (or your coroutines may be blocked
unnecessarily) and I think you have nothing except a more complicated
API to work with threads.

I think I am ready to offer a counterproposal where the event loop
runs in one thread and synchronous code runs in another thread and we
give the synchronous code a way to synchronously await a coroutine or
an asyncio.Future. This can be based on
asyncio.run_coroutine_threadsafe(), which takes a coroutine or an
asyncio.Future and returns a concurrent Future. (It also takes a loop,
and it assumes that loop runs in a different thread. I think it should
assert that.)

The main feature of my counterproposal as I see it is that async code
should not call back into synchronous code, IOW once you are writing
coroutines, you have to use the coroutine API for everything you do.
And if something doesn't have a coroutine API, you run it in a
background thread using loop.run_in_executor().

So either you buy into the async way of living and it's coroutines all
the way down from there, no looking back -- or you stay on the safe
side of the fence, and you interact with coroutines only using a very
limited "remote manipulator" API. The two don't mix any better than
that.

-- 
--Guido van Rossum (python.org/~guido)