[Python-ideas] The async API of the future: yield-from

[Guido van Rossum]

On Mon, Oct 15, 2012 at 4:37 PM, Matt Joiner <anacrolix at gmail.com> wrote:
> I gave something like this a go a while ago:
> https://bitbucket.org/anacrolix/green380
>
> "Coroutines" yield events or futures as Nick put them from the top, and the
> scheduler at the bottom manages events and scheduling.
>
> There are a few things I took away from this attempt:
>
> 1) Explicit yield a la PEP380 requires syntactical changes *everywhere*.

So does using PEP 342 style coroutines (yield Future instead of yield from).

> 2) Python's dynamic typing means that neglecting to "yield from" gives you
> broken code, and Python won't help you here. Add to this that you now have a
> 380, and "normal synchronous" form of most interfaces and the caller must
> know what kind is used at all times.

In NDB this is alleviated by insisting that the only thing you are
allowed to yield is a Future. Anything else raises TypeError. But yes,
the first few days when getting used to this style, you end up
debugging this a few times.

> 3) Concurrency is nice, but it requires language-level support, and proper
> parallelism to really shine. The "C way" of doing things is already so
> heavily ingrained in Python, an entirely new standard library and
> interpreter that breaks C compatibility is really the only way to proceed,
> and this certainly isn't worth it just to write code with "yield from"
> littered on every line.

Here you're basically arguing for greenlets/gevent -- you're saying
you just don't want to put up with the yields everywhere. But the
popularity of Twisted and Tornado show that at least some people are
willing to make even bigger sacrifices in order to be able to do async
I/O efficiently -- i.e., to solve the C10K problem that Christian
Tismer referred to (http://www.kegel.com/c10k.html,
http://en.wikipedia.org/wiki/C10k_problem).

There happen to be several problems with greenlets (even Christian
Tismer said so, and included Stackless in the problem). The current
effort is striving to help people solve it ith less effort than the
async style Twisted and Tornado promote, while avoiding the problems
with greenlets.

-- 
--Guido van Rossum (python.org/~guido)