[Python-ideas] Cofunctions - Getting away from the iterator protocol

[Ron Adam]

On Tue, 2011-11-08 at 15:46 +1000, Nick Coghlan wrote:
> On Tue, Nov 8, 2011 at 3:30 PM, Ron Adam <ron3200 at gmail.com> wrote:
> > Generator API.
> >
> >   Outside generator   <-->   Inside generator
> >
> >      next()                     yield
> >      .send()
> >      .throw()
> >
> >
> > Inverted generator API
> >
> >   Outside cothread    <-->   Inside cothread
> >
> >      .resume()                  suspend()
> >                                 throw()
> >
> > Where resume works like yield, (yield to cothread), and suspend() works
> > like .send().  Throw() raises an exception at the resume() call,
> > like .throw() raises an exception at the yield in a generator.
> 
> No, that doesn't make any sense.

Probably because I didn't explain it well enough.

> When the coroutine throws an
> exception internally it's done - we don't *want* to preserve the stack
> any more, because something broke and we won't be resuming it.

You mean throw as in a natural occurring exception rather than one
explicitly thrown. Different thing.

In the case of raise, (or throws due to an error.) true, but that's not
how throw() would work in an inverse generator API.  If we throw an
exception from the *inside*, it's not a coroutine error, it's re-raised
at the handler in the resume() call, not in the coroutine itself.  That
could work with generators as well.  Wish it did.

The reason that makes sense to do in coroutines is we most likely
already have a try except structure in the coroutine handler to catch
the exit and return status exceptions, so why not take advantage of that
and make it possibly for the coroutines to send out exceptions for other
things. with a throw() from inside the coroutine.  (and not unwind the
stack like a raise would.)

For the same reason you throw() an exception into a generator, you could
throw an exception out of a coroutine.  You don't *need* to do that with
either of them.  The alternative is to pass through the normal data
channel and parse, test, and/or filter it out once it gets to the other
side.  An try-except around a data input can be very efficient at doing
that with a lot less work.

> Instead, we let the exception bubble up the stack and if nothing
> handles it, we pass it back to the thread that called resume().

Right, and we can't resume from there in that case.

> The reason we need an explicit throw() is that the data request (or
> whatever it was we suspended to wait for) might fail - in that case,
> the thread calling resume() needs to be able to indicate this to the
> cothread by resuming with an exception.

Yes and no... Yes, that works, and no because it could work just as well
the other way around.

If generators had a throw keyword...

def adder(count):
   exc = None
   total = n = 0
   while n < count:
      try:
          if exc is None:
             x = yield 
          else:
             x = throw exc    # reraise exc in .send(), not here.
                              # suspends here, and waits for new x.
          total += x          # <-- error may be here.
          exc = None
          n += 1
      except Exception as e:
          exc = e
   yield total

In this case, the exception wouldn't bubble out, but be reraised at
the .send() where it can be handled.

I think that generators not being able to handle these types of things
gracefully is a reason not to use them with coroutines. A program based
on generator coroutines that you get an unexpected exception from needs
to be completely restarted. That makes sense for small iterators, but
not for larger programs.


> The flow control parallels are like this:
> 
>   Inside the generator/cothread:
>     yield -> cothread.suspend() # Wait to be resumed
>     return -> return # Finish normally
>     raise -> raise  # Bail out with an error
> 
>   Outside the generator/cothread
>     send() -> resume() # Resume execution normally (optionally providing data)
>     throw() -> throw() # Resume execution with an exception
> 
> Don't worry about next() in this context, since it's just an alternate
> spelling for send(None).

Yes, about the next.  And yes, this is the design in your cothread
module example. :)

Cheers,
   Ron