On 10/25/2010 03:21 PM, Guido van Rossum wrote:
On Mon, Oct 25, 2010 at 12:53 PM, Ron Adam
wrote: This is how my mind wants to write this.
@consumer def reduce_collector(func): try: value = yield # No value to yield here. while True: value = func((yield), value) # or here. except YieldError:
IIUC this works today if you substitute GeneratorExit and use c.close() instead of next(c) below. (I don't recall why I split it out into two different try/except blocks but it doesn't seem necessary.
I tried it, c.close() doesn't work yet, but it does work with c.throw(GeneratorExit) :-) But that still uses yield to get the value. I used a different way of starting the generator that checks for a value being yielded. class GeneratorStartError(TypeError): pass def start(g): value = next(g) if value is not None: raise GeneratorStartError('started generator yielded a value') return g def reduce_collector(func): value = None try: value = yield while True: value = func((yield), value) except GeneratorExit: yield value def parallel_reduce(iterable, funcs): collectors = [start(reduce_collector(func)) for func in funcs] for v in iterable: for coll in collectors: coll.send(v) return [c.throw(GeneratorExit) for c in collectors] def main(): it = range(100) print(parallel_reduce(it, [min, max])) if __name__ == '__main__': main()
As for being able to distinguish next(c) from c.send(None), that's a few language revisions too late. Perhaps more to the point, I don't like that idea; it breaks the general treatment of things that return None and throwing away values. (Long, long, long ago there were situations where Python balked when you threw away a non-None value. The feature was boohed off the island and it's better this way.)
I'm not sure I follow the relationship you suggest. No values would be thrown away. Or did you mean that it should be ok to throw away values? I don't think it would prevent that either. What the YieldError case really does is give the generator a bit more control. As far as the calling routine that uses it is concerned, it just works. What happend inside the generator is completely transparent to the routine using the generator. If the calling routine does see a YieldError, it means it probably was a bug.
# next was called not send. yield value
I object to overloading yield for both a *resumable* operation and returning a (final) value; that's why PEP 380 will let you write "return value". (Many alternatives were considered but we always come back to the simple "return value".)
That works for me. I think lot of people will find it easy to learn.
def parallel_reduce(iterable, funcs): collectors = [reduce_collector(func) for func in funcs] for v in iterable: for coll in collectors: coll.send(v) return [next(c) for c in collectors]
I really object to using next() for both getting the return value and the next yielded value. Jacob's proposal to spell this as c.close() sounds much better to me.
If c.close also throws the GeneratorExit and returns a value, that would be cool. Thanks. I take it that the objections have more to do with style and coding practices rather than what is possible.
It nicely separates input and output parts of a co-function, which can be tricky to get right when you have to receive and send at the same yield.
I don't think there was a problem with this in my code (or if there was you didn't solve it).
There wasn't in this code. This is one of those areas where it can be really difficult to find the correct way to express a co-function that does both input and output, but not necessarily in a fixed order. I begin almost any co-function with this at the top of the loop and later trim it up if parts of it aren't needed. out_value = None while True: in_value = yield out_value out_value = None ... # rest of loop to check in_value and modify out_value As long as None isn't a valid data item, this works most of the time.
Maybe in Python 4k? Oh well. :-)
Nah.
I'm ok with that. Ron