[Python-ideas] yield from multiple iterables (was Re: The async API of the future: yield-from)

[Nick Coghlan]

On Mon, Oct 15, 2012 at 11:57 PM, Christian Tismer <tismer at stackless.com> wrote:
> So in the context of switching, CPython is presumably more efficient
> than greenlet (because of stack slicing), and a bit less efficient than
> stackless because of the generator chaining.
>
> I have begun studying the code for YIELD_FROM. As it is written, every
> next iteration elevates the chain of generators once up and down.
> Maybe that can be avoided by changing the frame chain, so this can become
> a cheaper O(1) operation.

Yes, we certainly talked about that, but I don't believe anyone came
up with the code needed to make it behave itself properly when
unwinding the stack. (Either that or someone *did* try it, and then
undid it because it broke the test suite, which amounts to the same
thing. Mercurial could say for sure)

> Alternatively I could also imagine to write real generators or coroutines
> as an extension module. It would use the same concept as generators,
> internally. No big deal, not changing the interpreter, maybe adding a bit.

Tangentially related, there are some patches [1,2] on the tracker
looking to shuffle a few things related to generator state around to
get them out of the frame objects and into the generator objects where
they belong. There are definitely a few things that could do with
cleaning up in this space.

[1] http://bugs.python.org/issue13897
[2] http://bugs.python.org/issue13607

> I think this would make Greenlet and even Stackless obsolete in most
> cases which are of real use.

The "take this synchronous code and magically make it scale better"
aspect is still a nice feature of greenlets & gevent.

>
> I would like to discuss this and maybe do a prototype.

Sure, I think there's several things we can do better here, and I
think the test suite is comprehensive enough to keep us honest.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia