[Python-ideas] yield from multiple iterables (was Re: The async API of the future: yield-from)

[Greg Ewing]

Christian Tismer wrote:

> Right, CPython still keeps unneccessary crap on the C stack.

It's not just Python leaving stuff on the stack that's a
problem, it's external C code that calls back into Python.

> But that's not the point right now, because on the other hand,
> in the context of a possible yield (from or not), the C stack
> is clean, and this enables switching.

> And actually in such clean positions, Stackless Python (as opposed to
> Greenlets) does soft-switching, which is very similar to what the 
> generators
> are doing - there is no assembly stuff involved at all.

But the assembly code still needs to be there to handle the
cases where you *can't* do soft switching. It's the presence
of the code that's the issue here, not how frequently it
gets called.

> I have begun studying the code for YIELD_FROM. As it is written, every
> next iteration elevates the chain of generators once up and down.
> Maybe that can be avoided by changing the frame chain, so this can become
> a cheaper O(1) operation.

My original implementation of yield-from actually *did* avoid
this, by keeping a C-level pointer chain of yielding-from frames.
But that part was ripped out at the last minute when someone
discovered that it had a detrimental effect on tracebacks.

There are probably other ways the traceback problem could be
fixed, so maybe we will get this optimisation back one day.

-- 
Greg