[Python-ideas] yield from multiple iterables (was Re: The async API of the future: yield-from)

[Nick Coghlan]

On Mon, Oct 15, 2012 at 8:38 PM, Christian Tismer <tismer at stackless.com> wrote:
> Just one thing that I don't get.
> What do you mean by 'implicit taskswitching' ?
> There is no such thing in greenlet, if you really meant that
> Library from Armin Rigo.
>
> greenlets do everything explicitly, no pre-emption at all.
>
> So, is there a general understanding what a greenlet is and what not?
> Just to make sure that the discussed terms are clearly defined.

With greenlets, your potential switching points are every function
call (because you can call switch() from anywhere, and you can't
reliably know the name of *every* IO operation, or operation that
implicitly invokes an IO operation).

With generators, there is always an explicit *local* marker within the
generator body of the potential switching points: yield expressions
(including yield from). Ordinary function calls cannot cause the
function to be suspended.

So greenlets give you the scalability benefits of microthreading (as
almost any OS supports a couple of orders of magnitude more sockets
than it can threads), but without the same benefits of locally visible
suspension points that are provided by generators and explicit
callbacks.

That's the philosophical reason. As a *practical* matter, there's
still the problem you described in more detail elsewhere that CPython
relies too much on the C stack to support suspension of arbitrary call
chains without the stack switching assembly code in
Stackless/greenlets.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia