On Tue, May 5, 2015 at 1:44 PM, Paul Moore <p.f.moore@gmail.com> wrote:

[Guido]
> The run_in_executor call is not callback-based -- the confusion probably
> stems from the name of the function argument ('callback'). It actually
> returns a Future representing the result (or error) of an operation, where
> the operation is represented by the function argument. So if you have e.g. a
> function
>
> def factorial(n):
> return 1 if n <= 0 else n*factorial(n-1)
>
> you can run it in an executor from your async(io) code like this:
>
> loop = asyncio.get_event_loop()
> result = yield from loop.run_in_executor(factorial, 100)
>
> (In a PEP 492 coroutine substitute await for yield from.)

Thanks, that's an important correction. Given that, run_in_executor is
the link to blocking calls that I was searching for. And yes, the
"callback" terminology does make this far from obvious, unfortunately.
As does the point at which it's introduced (before futures have been
described) and the fact that it says "this method is a coroutine"
rather than "this method returns a Future"[1].

Paul

[1] I'm still struggling to understand the terminology, so if those
two statements are equivalent, that's not yet obvious to me.

I apologize for the confusing documentation. We need more help from qualified tech writers! Writing PEP 3156 was a huge undertaking for me; after that I was exhausted and did not want to take on writing the end user documentation as well, so it was left unfinished. :-(

In PEP 3156 (asyncio package) there are really three separate concepts:

- Future, which is a specific class (of which Task is a subclass);

- coroutine, by which in this context is meant a generator object obtained by calling a generator function decorated with @asyncio.coroutine and written to conform to the asyncio protocol for coroutines (i.e. don't use bare yield, only use yield from, and the latter always with either a Future or a coroutine as argument);

- either of the above, which is actually the most common requirement -- most asyncio functions that support one also support the other, and either is allowable as the argument to `yield from`.

In the implementation we so often flipped between Future and coroutine that I imagine sometimes the implementation and docs differ; also, we don't have a good short name for "either of the above" so we end up using one or the other as a shorthand.

*Unless* you want to attach callbacks, inspect the result or exception, or cancel it (all of which require a Future), your code shouldn't be concerned about the difference -- you should just use `res = yield from func(args)` and use try/except to catch exceptions if you care. And if you do need a Future, you can call the function asyncio.async() on it (which in PEP 492 is renamed to ensure_future()).

In the PEP 492 world, these concepts map as follows:

- Future translates to "something with an __await__ method" (and asyncio Futures are trivially made compliant by defining Future.__await__ as an alias for Future.__iter__);

- "asyncio coroutine" maps to "PEP 492 coroutine object" (either defined with `async def` or a generator decorated with @types.coroutine -- note that @asyncio.coroutine incorporates the latter);

- "either of the above" maps to "awaitable".

--Guido van Rossum (python.org/~guido)