[Python-Dev] PEP 492: async/await in Python; version 4

[Guido van Rossum]

On Tue, May 5, 2015 at 1:44 PM, Paul Moore <p.f.moore at gmail.com> wrote:

> [Guido]
> > The run_in_executor call is not callback-based -- the confusion probably
> > stems from the name of the function argument ('callback'). It actually
> > returns a Future representing the result (or error) of an operation,
> where
> > the operation is represented by the function argument. So if you have
> e.g. a
> > function
> >
> >     def factorial(n):
> >         return 1 if n <= 0 else n*factorial(n-1)
> >
> > you can run it in an executor from your async(io) code like this:
> >
> >     loop = asyncio.get_event_loop()
> >     result = yield from loop.run_in_executor(factorial, 100)
> >
> > (In a PEP 492 coroutine substitute await for yield from.)
>
> Thanks, that's an important correction. Given that, run_in_executor is
> the link to blocking calls that I was searching for. And yes, the
> "callback" terminology does make this far from obvious, unfortunately.
> As does the point at which it's introduced (before futures have been
> described) and the fact that it says "this method is a coroutine"
> rather than "this method returns a Future"[1].
>
> Paul
>
> [1] I'm still struggling to understand the terminology, so if those
> two statements are equivalent, that's not yet obvious to me.
>

I apologize for the confusing documentation. We need more help from
qualified tech writers! Writing PEP 3156 was a huge undertaking for me;
after that I was exhausted and did not want to take on writing the end user
documentation as well, so it was left unfinished. :-(

In PEP 3156 (asyncio package) there are really three separate concepts:

- Future, which is a specific class (of which Task is a subclass);

- coroutine, by which in this context is meant a generator object obtained
by calling a generator function decorated with @asyncio.coroutine and
written to conform to the asyncio protocol for coroutines (i.e. don't use
bare yield, only use yield from, and the latter always with either a Future
or a coroutine as argument);

- either of the above, which is actually the most common requirement --
most asyncio functions that support one also support the other, and either
is allowable as the argument to `yield from`.

In the implementation we so often flipped between Future and coroutine that
I imagine sometimes the implementation and docs differ; also, we don't have
a good short name for "either of the above" so we end up using one or the
other as a shorthand.

*Unless* you want to attach callbacks, inspect the result or exception, or
cancel it (all of which require a Future), your code shouldn't be concerned
about the difference -- you should just use `res = yield from func(args)`
and use try/except to catch exceptions if you care. And if you do need a
Future, you can call the function asyncio.async() on it (which in PEP 492
is renamed to ensure_future()).

In the PEP 492 world, these concepts map as follows:

- Future translates to "something with an __await__ method" (and asyncio
Futures are trivially made compliant by defining Future.__await__ as an
alias for Future.__iter__);

- "asyncio coroutine" maps to "PEP 492 coroutine object" (either defined
with `async def` or a generator decorated with @types.coroutine -- note
that @asyncio.coroutine incorporates the latter);

- "either of the above" maps to "awaitable".

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150505/08304987/attachment.html>