[Python-ideas] The async API of the future: yield-from

[Dino Viehland]

Guido wrote:
> But do your Futures use threads? NDB doesn't. NDB's event loop doesn't
> know about Futures; however the @ndb.tasklet decorator does, and the
> Futures know about the event loop. When you wait for a Future, a callback is
> added to the Future that will resume the generator when it is done, and in
> order to run them, the Future passes its callbacks to the event loop to be
> run.

The decorator and the default context don't do anything w/ threads by default,
but once you start combining it w/ other futures threads are likely to be used.
For example if you take:

         @async
         def get_image_async(url):
             buffer = yield executor.submit(load_url, url)
             return Image(buffer)

Then the " yield executor.submit(load_url, url)" line is going to yield a future which
is running on a thread pool thread.  When it completes it's done callback is also going
to be delivered on the same thread pool thread.  At that point we let the context which
was captured when the function was initially called handle resuming the generator.  The
default context is just going to synchronously continue to the function, so the
generator would then resume running on the thread pool thread.  But if you're running
in a GUI app which sets up its own context then the context will post an event into the UI 
event loop and execution will continue on the UI thread.

Likewise if there were a bunch of async I/O routines then this would combine with them in 
a similar way - async I/O would result in a future, the futures would signal that they're done 
on some worker thread, and then the async methods will get to continue running on that 
worker thread unless the current context wants to do something different.