It does seem like there are two different use cases for generators which the return value semantics of "yield from" starts to make very clear. The original use is building an iterator which produces results in sequence, and the other is the "cofunction" style of executing a sequence of code blocks which might be interleaved with code from the caller and which returns a value like a regular function. I find cofunctions to be a very useful concept.
I like the idea of "codef" defining these cofunctions in a way which makes them behave like generators even if they do not contain a yield call. Certainly having to revisit call sites when you shuffle or comment out code is troublesome.
What I am suspicious of is automatically making calls to cofunctions from within a cofunction imply 'yield from'. Knowing whether the function call you are making can suspend your code and return to your parent or not is important, since your state might change between when you call some function and when it returns. Instead, I would like to see a different calling mechanism when calling a cofunction, which makes it clear that you know you're calling a cofunction and that this could have implications for your own code. "yield" (and "yield from") serve this purpose somewhat. It seems like calling a normal function using this special calling mechanism could treat them like a cofunction which produced zero iterations and a single return value.
Obviously this topic is very close to the monocle framework ( http://github.com/saucelabs/monocle ), where we implement something very much like this as best we can using existing Python functionality. Other projects have tried to use C module stack-swapping coroutines, and I have seen programmers (including the developers of these projects) struggle with the unexpected, thread-like preemption. I believe a word like "yield" makes these cooperation points more obvious.
-Greg
On Sun, Aug 1, 2010 at 3:09 AM, Greg Ewing greg.ewing@canterbury.ac.nz wrote:
I've been thinking about this idea for a while, and now that I've got yield-from nailed down to my satisfaction, I thought I'd put it out there to see if anyone else thinks it would be a good thing.
Cofunctions
A drawback of 'yield from' as a coroutine mechanism is that it must be used every time a function forming part of a coroutine calls another function that can suspend the coroutine, either directly or indirectly.
This makes the code read somewhat awkwardly and provides many opportunities for the programmer to make errors. It also introduces considerable coupling, since changing one's mind about whether a function needs to be suspendable requires revisiting all the call sites of that function.
This proposal builds on the 'yield from' proposal by introducing a new kind of function that I will call a "cofunction".
A cofunction is a special kind of generator, with the following characteristics:
It is defined by using the keyword 'codef' in place of 'def'.
It is always a generator, even if it does not contain any yields.
Whenever a call is made inside a cofunction, it is done using a
special COCALL opcode. This first looks for a __cocall__ method on the object being called. If present, it is expected to return an iterable object, which is treated as though 'yield from' had been performed on it.
If the object being called does not have a __cocall__ method, or it returns NotImplemented, the call is made in the usual way through the __call__ method.
- Cofunctions themselves have a __cocall__ method that does the
same thing as __call__.
Using these cofunctions, it should be possible to write coroutine code that looks very similar to ordinary code. Cofunctions can call both ordinary functions and other cofunctions using ordinary call syntax. The only special consideration is that 'codef' must be used to define any function that directly or indirectly calls another cofunction.
A few subtle details:
- Ordinary generators will *not* implement __cocall__. This is so
that a cofunction can e.g. contain a for-loop that iterates over a generator without erroneously triggering yield-from behaviour.
- Some objects that wrap functions will need to be enhanced with
__cocall__ methods that delegate to the underlying function. Obvious ones that spring to mind are bound methods, staticmethods and classmethods.
Returning NotImplemented is specified as one of the possible responses of __cocall__ so that a wrapper can report that the wrapped object does not support __cocall__.
-- Greg
Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas