[Python-ideas] Linking generators

11 Dec 2010

      On 10/27/2010 11:53 AM, Jacob Holm wrote:
...
On 2010-10-26 18:56, Guido van Rossum wrote:
...
Now, if I may temporarily go into wild-and-crazy mode (this *is*
python-ideas after all :-), we could invent some ad-hoc syntax for
this pattern, e.g.:
...
Hmm.  This got me thinking.  One thing I'd really like to see in python
is something like the "channel" object from the go language
(http://golang.org/).
Based on PEP 380 or Gregs new cofunctions PEP (or perhaps even without
any of them) it is possible to write a trampoline-based implementation
of a channel object with "send" and "next" methods that work as
expected.  One thing that is *not* possible (I think) is to make that
object iterable.  Your wild idea above gave me a similar wild idea of my
own.  An extension to the cofunctions PEP that would make that possible.
...
[clipped]
...
All this to make it possible to write a code like this:
def consumer(ch):
     for val in ch:
         cocall print(val) # XXX need a cocall somewhere
def producer(ch):
     for val in range(10):
         cocall ch.send(val)
def main()
     sched = scheduler()
     ch = channel()
     sched.add(consumer(ch))
     sched.add(producer(ch))
     sched.run()
Thoughts?
The following isn't quite the same as channels, but you might be able to 
use the technique below to get something like it.

These examples show how linking generators using nonlocal can simplify some 
problems. The nice thing about these is the inner loops are very simple and 
there is no try-except blocks, exceptions, or if statements involved. ;-)

Ron

         def averager():

             def collector():
                 nonlocal tally, count
                 while 1:
                     tally += yield
                     count += 1

             def emitter():
                 nonlocal tally, count
                 while 1:
                     yield tally / count

             count = tally = 0
             coll = collector()
             next(coll)
             return coll, emitter()

         coll, emit = averager()
         for x in range(100):
             coll.send(x)
         print('average: %s' % next(emit))

         def parallel_reduce(iterable, funcs):

             def reduce_collector(func):

                 def collector(func):
                     nonlocal outcome
                     outcome = yield
                     while 1:
                         outcome = func(outcome, (yield))

                 def emitter():
                     nonlocal outcome
                     while 1:
                         yield outcome

                 outcome = None
                 coll = collector(func)
                 next(coll)
                 return coll, emitter()

             collectors = [reduce_collector(func) for func in funcs]
             for val in iterable:
                 for coll, _ in collectors:
                     coll.send(val)
             return [next(emit) for _, emit in collectors]

         print(parallel_reduce(range(5, 10), [min, max]))

[Python-ideas] Linking generators

Ron Adam