On Mon, Oct 25, 2010 at 6:01 PM, Ron Adam
On 10/25/2010 03:21 PM, Guido van Rossum wrote:
On Mon, Oct 25, 2010 at 12:53 PM, Ron Adam
wrote: This is how my mind wants to write this.
@consumer def reduce_collector(func): try: value = yield # No value to yield here. while True: value = func((yield), value) # or here. except YieldError:
IIUC this works today if you substitute GeneratorExit and use c.close() instead of next(c) below. (I don't recall why I split it out into two different try/except blocks but it doesn't seem necessary.
I tried it, c.close() doesn't work yet, but it does work with c.throw(GeneratorExit) :-) But that still uses yield to get the value.
Yeah, sorry, I didn't mean to say that g.close() would return the value, but that you can use GeneratorExit here. g.close() *does* throw GeneratorExit (that's PEP 342); but it doesn't return the value yet. I like adding that to PEP 380 though.
I used a different way of starting the generator that checks for a value being yielded.
class GeneratorStartError(TypeError): pass
def start(g): value = next(g) if value is not None: raise GeneratorStartError('started generator yielded a value') return g
Whatever tickles your fancy. I just don't think this deserves a builtin.
def reduce_collector(func): value = None try: value = yield while True: value = func((yield), value) except GeneratorExit: yield value
Even today, I would much prefer using raise StopIteration(value) over yield value (or yield Return(value)). Reusing yield to return a value just looks wrong to me, there are too many ways to get confused (and this area doesn't need more of that :-).
def parallel_reduce(iterable, funcs): collectors = [start(reduce_collector(func)) for func in funcs] for v in iterable: for coll in collectors: coll.send(v) return [c.throw(GeneratorExit) for c in collectors]
def main(): it = range(100) print(parallel_reduce(it, [min, max]))
if __name__ == '__main__': main()
As for being able to distinguish next(c) from c.send(None), that's a few language revisions too late. Perhaps more to the point, I don't like that idea; it breaks the general treatment of things that return None and throwing away values. (Long, long, long ago there were situations where Python balked when you threw away a non-None value. The feature was boohed off the island and it's better this way.)
I'm not sure I follow the relationship you suggest. No values would be thrown away. Or did you mean that it should be ok to throw away values? I don't think it would prevent that either.
Well maybe I was misunderstanding your proposed YieldError. You didn't really explain it -- you just used it and assumed everybody understood what you meant. My assumption was that you meant for YieldError to be raised if yield was used as an expression (not a statement) but next() was called instead of send(). My response was that it's ugly to make a distinction between x = <expr> del x # Or just not use x and <expr> But maybe I misunderstood what you meant.
What the YieldError case really does is give the generator a bit more control. As far as the calling routine that uses it is concerned, it just works. What happend inside the generator is completely transparent to the routine using the generator. If the calling routine does see a YieldError, it means it probably was a bug.
That sounds pretty close to the rules for GeneratorExit.
# next was called not send. yield value
I object to overloading yield for both a *resumable* operation and returning a (final) value; that's why PEP 380 will let you write "return value". (Many alternatives were considered but we always come back to the simple "return value".)
That works for me. I think lot of people will find it easy to learn.
def parallel_reduce(iterable, funcs): collectors = [reduce_collector(func) for func in funcs] for v in iterable: for coll in collectors: coll.send(v) return [next(c) for c in collectors]
I really object to using next() for both getting the return value and the next yielded value. Jacob's proposal to spell this as c.close() sounds much better to me.
If c.close also throws the GeneratorExit and returns a value, that would be cool. Thanks.
It does throw GeneratorExit (that's the whole reason for GeneratorExit's existence :-).
I take it that the objections have more to do with style and coding practices rather than what is possible.
Yeah, it's my gut and that's hard to reason with but usually right. (See also: http://www.amazon.com/How-We-Decide-Jonah-Lehrer/dp/0618620117 )
It nicely separates input and output parts of a co-function, which can be tricky to get right when you have to receive and send at the same yield.
I don't think there was a problem with this in my code (or if there was you didn't solve it).
There wasn't in this code. This is one of those areas where it can be really difficult to find the correct way to express a co-function that does both input and output, but not necessarily in a fixed order.
Maybe for that one should use a "channel" abstraction, like Go (and before it, CSP)? I noticed that Monocle (http://github.com/saucelabs/monocle) has a demo of that in its "experimental" module (but the example is kind of silly).
I begin almost any co-function with this at the top of the loop and later trim it up if parts of it aren't needed.
out_value = None while True: in_value = yield out_value out_value = None ... # rest of loop to check in_value and modify out_value
As long as None isn't a valid data item, this works most of the time.
Maybe in Python 4k? Oh well. :-)
Nah.
I'm ok with that.
Ron
-- --Guido van Rossum (python.org/~guido)