[Python-ideas] Yield-from: Suspendable generators
greg.ewing at canterbury.ac.nz
Fri Feb 20 04:09:25 CET 2009
Whether you're using yield-from or not, it doesn't
seem to be possible to have a for-loop iterating
over something that is also suspendable in a
generator-based thread setting.
The basic problem is that we have one channel and
two different things we want to use it for. The
obvious answer is that we need to multiplex. Since
we're already using the entire bandwidth of the
channel (anything could be a valid yielded value)
we need to introduce some out-of-band data somehow.
Suppose we have a new expression
suspend [<value>] [with <tag>]
This is a lot like a yield, except that it sends
a tuple (value, tag).
The existing yield expression
becomes equivalent to
suspend <value> with 'yield'
There is a new generator method to go along with
If the generator is suspended at a suspend expression,
the value of the suspend expression becomes (value, tag).
If it is suspended at a yield, and the tag is 'yield'
then the value becomes the value of the yield expression.
(Not sure what to do in other cases, maybe raise an
The existing send() method is mapped to resume() as
def send(self, value):
value2, tag2 = self.resume(value, 'yield')
if tag2 == 'yield':
# What to do here? Ignore? Raise an exception?
So we've generalised the yield channel into a suspend
channel, which can have any number of sub-channels. We have
reserved one of these sub-channels, tagged with 'yield', for
carrying yielded values. The rest of the channels are free
for use by other things such as thread-scheduling libraries.
To complete this, we also need a variant of the for-loop
that is willing to pass values from the other channels on to
the caller. Picking a random syntax for illustration,
for y from g(x):
would be roughly equivalent to
it = g(x)
value = None
tag = 'yield'
value, tag = it.resume(value, tag)
if tag == 'yield':
y = value
value = None
value, tag = suspend value with tag
plus suitable handling of 'throw' and 'close'.
More information about the Python-ideas