[Python-ideas] Cofunctions/yield from -> fibers

[Ivan Pozdeev]

The original proposal for introducing 'yield from' (which is still
in PEP 380's name) was to delegate a part of generator's work to another generator.
However, in later discussions, the focus totally shifted to
cooperative multitasking.

In the example code Greg has given in
http://mail.python.org/pipermail/python-ideas/2010-August/007927.html ,

there's not a single use case for delegating! 'yield from's essentially replace
normal function calls!
All Greg uses this stuff for is to manually switch between `threads'
simulating individual customers! The _programming_logic_ is plain function calls,
yields/cocalls are just a crutch (and quite an ugly one) to release a time slot.


So, in fact, this all `yield from/coroutine' effort is an attempt to
unseparatably mix two very loosely-related subjects:

- code delegation (sequence in which sequential/closely-related code is executed)
- execution scheduling (sequence in which parallel/loosely-related/unrelated code is executed)

in the form 'a function call _may_be_ a scheduling event at the same time'!


That's why it all feels so `clumsy/weird/twisted/fallacious' !
Cooperative threads must be switched somehow but choosing such a
quirky and implicit technique for that is completely against Python
Zen (violates about half of the items :^) )!


Instead, if it's cooperative multitasking you play with,
switching must be independent from other activity and as explicit as possible.
There's a technique just for that called 'fibers'
(MS fiber API: http://msdn.microsoft.com/en-us/library/ms682661.aspx ).

In short:
- ConvertThreadToFiber() decorates current thread as an initial fiber
- CreateFiber (*function, *parameter)->ID creates a new fiber
- SwitchToFiber(ID) switches execution to another fiber.
- ExitFiber() exits a fiber. In python, the function may just return
as in threads

As the goal of fibers is the same as threads, it's reasonable to
derive knowledge from there too, maybe duplicate the interface
where applicable.

And as cofunctions, these Fibers are just what
http://en.wikipedia.org/wiki/Coroutine describes. With interacting coroutines,
there is no execution stack anymore: there are 'calls' but no 'returns'!
It's essentially like passing messages back and forth as in win32k.


`Yield from's are still valid. But only as code delegation technique, i.e. a
shortcut to `for i in subgen(): yield i'.
The latter statement looks brief enough for me to discard the proposal
altogether. Send() and stuff doesn't look to fit in what
generators were originally intended to do - producing sequences with
arbitrary or unknown length.