[Python-ideas] x=(yield from) confusion [was:Yet another alternative name for yield-from]
Jacob Holm
jh at improva.dk
Tue Apr 7 17:00:27 CEST 2009
Greg Ewing wrote:
> Jacob Holm wrote:
>
>> One major reason for factoring something out of a coroutine would be
>> if the factored-out code was independently useful as a coroutine.
>
> So provide another entry point for using that part
> as a top level, or manually apply the wrapper when
> it's appropriate.
It is inconvenient to have to keep separate versions around, and it
doesn't solve the problem. See example below.
>
> I find it extroardinary that people seem to have
> latched onto David Beazley's idiosyncratic definition
> of a "coroutine" and decided that it's written on
> a stone tablet from Mt. Sinai that we must always
> wrap them in his decorator.
I find it extraordinary that my critical perspective on this issue makes
you think I am taking his tutorial as dogma. There is a real issue here
IMNSHO, and the @coroutine examples just happens to be the easiest way I
can see of explaining it.
>
>> @coroutine
>> def avg_diff():
>> a = yield from avg2() start None # "start EXPR" means use EXPR
>> for first value to yield instead of next()
>> b = yield from avg2() start a # "start EXPR" means use EXPR
>> for first value to yield instead of next()
>
> I'm going to need a less abstract example to see
> why you might want to do something like that.
>
Ok, below you will find a modified version of your parser example taken
from
http://www.cosc.canterbury.ac.nz/greg.ewing/python/yield-from/parser.txt
The modification consists of applying the @coroutine decorator to
parse_items and parse_elem and changing them to yield a stream of
2-tuples describing how each token sent to the coroutine was interpreted.
The expected output is:
Feeding: '<foo>'
Yielding: ('Open', 'foo')
Feeding: 'This'
Yielding: ('Data', 'This')
Feeding: 'is'
Yielding: ('Data', 'is')
Feeding: 'a'
Yielding: ('Data', 'a')
Feeding: '<b>'
Yielding: ('Open', 'b')
Feeding: 'foo'
Yielding: ('Data', 'foo')
Feeding: 'file'
Yielding: ('Data', 'file')
Feeding: '</b>'
Yielding: ('Close', 'b')
Feeding: 'you'
Yielding: ('Data', 'you')
Feeding: 'know.'
Yielding: ('Data', 'know.')
Feeding: '</foo>'
Yielding: ('Close', 'foo')
[('foo', ['This', 'is', 'a', ('b', ['foo', 'file']), 'you', 'know.'])]
I can't see a nice way to get the same sequence of send() calls to yield
the same values without the ability to override the way the value for
the initial yield in yield-from is computed. Even avoiding the use of
@coroutine, I will still need to pass extra arguments to the generator
functions to control the initial yield.
- Jacob
------------------------------------------------------------------------
# Support code from example at http://www.cosc.canterbury.ac.nz/greg.ewing/python/yield-from/parser.txt
import re
pat = re.compile(r"(\S+)|(<[^>]*>)")
def scanner(text):
for m in pat.finditer(text):
token = m.group(0)
print "Feeding:", repr(token)
yield token
yield None # to signal EOF
text = "<foo> This is a <b> foo file </b> you know. </foo>"
token_stream = scanner(text)
def is_opening_tag(token):
return token.startswith("<") and not token.startswith("</")
# Coroutine decorator copied from earlier mail, based on the one in http://dabeaz.com/coroutines/
def coroutine(func):
def start(*args, **kwargs):
cr = func(*args, **kwargs)
v = cr.next()
if v is not None:
raise RuntimeError('first yield from coroutine was not None')
return cr
return start
# Runner modified from example at http://www.cosc.canterbury.ac.nz/greg.ewing/python/yield-from/parser.txt
# to also print the yielded values.
def run():
parser = parse_items()
# The original forgot to call next() here. That is not necessary in this version since parse_items uses
# the @coroutine decorator.
try:
for m in pat.finditer(text):
token = m.group(0)
print "Feeding:", repr(token)
v = parser.send(token)
print "Yielded:", v
parser.send(None) # to signal EOF
except StopIteration, e:
tree = e.args[0]
print tree
# parse_elem modified from example at http://www.cosc.canterbury.ac.nz/greg.ewing/python/yield-from/parser.txt
# to make it a coroutine and to yield a sequence of 2-tuples describing how the recieved data was interpreted.
# (Does not yield the final ('Close', <tagname>) because it returns the tree instead).
@coroutine
def parse_elem():
opening_tag = yield
name = opening_tag[1:-1]
closing_tag = "</%s>" % name
items = yield from parse_items(closing_tag) start ('Open', name)
return (name, items)
# parse_items modified from example at http://www.cosc.canterbury.ac.nz/greg.ewing/python/yield-from/parser.txt
# to make it a coroutine and to yield a sequence of 2-tuples describing how the recieved data was interpreted.
@coroutine
def parse_items(closing_tag = None):
elems = []
token = yield
while token != closing_tag:
if is_opening_tag(token):
subtree = yield from parse_elem() as p start p.send(token)
elems.append(subtree)
out = ('Close', subtree[0])
else:
elems.append(token)
out = ('Data', token)
token = yield out
return elems
More information about the Python-ideas
mailing list