[Python-ideas] yield from multiple iterables (was Re: The async API of the future: yield-from)

[Christian Tismer]

On 19.10.12 15:56, Nick Coghlan wrote:
> On Fri, Oct 19, 2012 at 10:55 PM, Christian Tismer <tismer at stackless.com> wrote:
>> I was curious and searched quite a lot.
>> It was v3.3.0a1 from 2012-03-15 as a reaction to #14230 and #14220
>> from Marc Shannon, patched by Benjamin.
>>
>> Now I found the original implementation. That looks very much
>> as I'm thinking it should be.
>>
>> Quite a dramatic change which works well, but really seems to remove
>> what I would call "now I can emulate most of Stackless" efficiently.
>>
>> Maybe I should just try to think it would be implemented as before,
>> build an abstraction and just use it for now.
>>
>> I will spend my time at PyCon de for sprinting on "yield from".
> Yeah, if we can get Greg's original optimised behaviour while still
> supporting introspection properly, that's really where we want to be.
> That's the main reason I'm a fan of Mark's other patches moving more
> of the generator state from the frame objects out into the generator
> objects - my suspicion is that generator objects themselves need to be
> maintaining a full "generator stack" independent of the frame stack in
> the main eval loop in order to get the best of both worlds (i.e.
> optimised suspend/resume with confusing debuggers).

That may be very true in order to get real generators.
The storm in my brain is quite intense the last days...

Actually I would like to have a python context where it gets into
"async mode" and interprets all functions defined in that mode as 
generators.
In that mode, generators are not meant as generators, but async-enabled
functions.
I see "yield from" as a low-level construct that should not even be
exposed, but be applied automatically in async mode.
That way, we could write normal functions and could implement a real "Yield"
without the "yield from" helper visible everywhere.

Not sure how to do that right. I'm playing with AST a bit to get a feeling
for this.

To give you an idea where my thoughts are meandering around,
I would like to point you at

http://doc.pypy.org/en/latest/stackless.html

That is an implementation that comes close to what I'm thinking.
The drawback of the current PyPy implementation is that it
used greenlet style for its underlying switching. That is what
I want to replace with some "yield from" construct.

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
Software Consulting          :     Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776  fax +49 (30) 700143-0023
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/