[Python-ideas] Possible PEP 380 tweak

[Ron Adam]

On 10/25/2010 03:21 PM, Guido van Rossum wrote:
> On Mon, Oct 25, 2010 at 12:53 PM, Ron Adam<rrr at ronadam.com>  wrote:
>> This is how my mind wants to write this.
>>
>> @consumer
>> def reduce_collector(func):
>>     try:
>>         value = yield            # No value to yield here.
>>         while True:
>>             value = func((yield), value)        # or here.
>>     except YieldError:
>
> IIUC this works today if you substitute GeneratorExit and use
> c.close() instead of next(c) below. (I don't recall why I split it out
> into two different try/except blocks but it doesn't seem necessary.

I tried it, c.close() doesn't work yet, but it does work with 
c.throw(GeneratorExit) :-)   But that still uses yield to get the value.

I used a different way of starting the generator that checks for a value 
being yielded.



class GeneratorStartError(TypeError): pass

def start(g):
     value = next(g)
     if value is not None:
         raise GeneratorStartError('started generator yielded a value')
     return g

def reduce_collector(func):
     value = None
     try:
         value = yield
         while True:
             value = func((yield), value)
     except GeneratorExit:
         yield value

def parallel_reduce(iterable, funcs):
     collectors = [start(reduce_collector(func)) for func in funcs]
     for v in iterable:
         for coll in collectors:
             coll.send(v)
     return [c.throw(GeneratorExit) for c in collectors]

def main():
     it = range(100)
     print(parallel_reduce(it, [min, max]))

if __name__ == '__main__':
     main()



> As for being able to distinguish next(c) from c.send(None), that's a
> few language revisions too late. Perhaps more to the point, I don't
> like that idea; it breaks the general treatment of things that return
> None and throwing away values. (Long, long, long ago there were
> situations where Python balked when you threw away a non-None value.
> The feature was boohed off the island and it's better this way.)

I'm not sure I follow the relationship you suggest.  No values would be 
thrown away.  Or did you mean that it should be ok to throw away values? I 
don't think it would prevent that either.

What the YieldError case really does is give the generator a bit more 
control.  As far as the calling routine that uses it is concerned, it just 
works.  What happend inside the generator is completely transparent to the 
routine using the generator.  If the calling routine does see a YieldError, 
it means it probably was a bug.


>>         # next was called not send.
>>         yield value
>
> I object to overloading yield for both a *resumable* operation and
> returning a (final) value; that's why PEP 380 will let you write
> "return value". (Many alternatives were considered but we always come
> back to the simple "return value".)

That works for me.  I think lot of people will find it easy to learn.


>> def parallel_reduce(iterable, funcs):
>>     collectors = [reduce_collector(func) for func in funcs]
>>     for v in iterable:
>>         for coll in collectors:
>>             coll.send(v)
>>     return [next(c) for c in collectors]
>
> I really object to using next() for both getting the return value and
> the next yielded value. Jacob's proposal to spell this as c.close()
> sounds much better to me.

If c.close also throws the GeneratorExit and returns a value, that would be 
cool. Thanks.

I take it that the objections have more to do with style and coding 
practices rather than what is possible.


>> It nicely separates input and output parts of a co-function, which can be
>> tricky to get right when you have to receive and send at the same yield.
>
> I don't think there was a problem with this in my code (or if there
> was you didn't solve it).

There wasn't in this code.  This is one of those areas where it can be 
really difficult to find the correct way to express a co-function that does 
both input and output, but not necessarily in a fixed order.

I begin almost any co-function with this at the top of the loop and later 
trim it up if parts of it aren't needed.

    out_value = None
    while True:
       in_value = yield out_value
       out_value = None
       ...
       # rest of loop to check in_value and modify out_value

As long as None isn't a valid data item, this works most of the time.


>> Maybe in Python 4k?  Oh well. :-)
>
> Nah.

I'm ok with that.

Ron