[Python-Dev] PEP 342 suggestion: start(), __call__() and unwind_call() methods

[Nick Coghlan]

Phillip J. Eby wrote:
> At 09:50 PM 10/7/2005 +1000, Nick Coghlan wrote:
> 
>> Notice how a non-coroutine callable can be yielded, and it will still 
>> work
>> happily with the scheduler, because the desire to continue execution is
>> indicated by the ContinueIteration exception, rather than by the type 
>> of the
>> returned value.
> 
> 
> Whaaaa?  You raise an exception to indicate the *normal* case?  That 
> seems, um...  well, a Very Bad Idea.

The sheer backwardness of my idea occurred to me after I'd got some sleep :)

> Last, but far from least, as far as I can tell you can implement all of 
> these semantics using PEP 342 as it sits.  That is, it's very simple to 
> make decorators or classes that add those semantics.  I don't see 
> anything that requires them to be part of Python.

Yeah, I've now realised that you can do all of this more simply by doing it 
directly in the scheduler using StopIteration to indicate when the coroutine 
is done, and using yield to indicate "I'm not done yet".

So with a bit of thought, I came up with a scheduler that has all the benefits 
I described, and only uses the existing PEP 342 methods.

When writing a coroutine for this scheduler, you can do 6 things via the 
scheduler:

   1. Raise StopIteration to indicate "I'm done" and return None to your caller
   2. Raise StopIteration with a single argument to return a value other than 
None to your caller
   3. Raise a different exception and have that exception propagate up to your 
caller
   5. Yield None to allow other coroutines to be executed
   5. Yield a coroutine to request a call to that coroutine
   6. Yield a callable to request an asynchronous call using that object

Yielding anything else, or trying to raise StopIteration with more than one 
argument results in a TypeError being raised *at the point of the offending 
yield or raise statement*, rather than taking out the scheduler itself.

The more I explore the possibilities of PEP 342, the more impressed I am by 
the work that went into it!

Cheers,
Nick.

P.S. Here's the Trampoline scheduler described above:

         import collections

         class Trampoline:
             """Manage communications between coroutines"""

             running = False

             def __init__(self):
                 self.queue = collections.deque()

             def add(self, coroutine):
                 """Request that a coroutine be executed"""
                 self.schedule(coroutine)

             def run(self):
                 result = None
                 self.running = True
                 try:
                     while self.running and self.queue:
                         func = self.queue.popleft()
                         result = func()
                     return result
                 finally:
                     self.running = False

             def stop(self):
                 self.running = False

             def schedule(self, coroutine, stack=(), call_result=None, *exc):
                 # Define the new pseudothread
                 def pseudothread():
                     try:
                         if exc:
                             callee = coroutine.throw(call_result, *exc)
                         else:
                             callee = coroutine(call_result)
                     except (StopIteration), ex:
                         # Coroutine finished cleanly
                         if stack:
                             # Send the result to the caller
                             caller = stack[0]
                             prev_stack = stack[1]
                             if len(ex.args) > 1:
                                 # Raise a TypeError in the current coroutine
                                 self.schedule(coroutine, stack,
                                      TypeError,
                                      "Too many arguments to StopIteration"
                                 )
                             elif ex.args:
                                 self.schedule(caller, prev_stack, ex.args[0])
                             else:
                                 self.schedule(caller, prev_stack)
                     except:
                         # Coroutine finished with an exception
                         if stack:
                             # send the error back to the caller
                             caller = stack[0]
                             prev_stack = stack[1]
                             self.schedule(
                                  caller, prev_stack, *sys.exc_info()
                             )
                         else:
                             # Nothing left in this pseudothread to
                             # handle it, let it propagate to the
                             # run loop
                             raise
                     else:
                         # Coroutine isn't finished yet
                         if callee is None:
                             # Reschedule the current coroutine
                             self.schedule(coroutine, stack)
                         elif isinstance(callee, types.GeneratorType):
                             # Requested a call to another coroutine
                             self.schedule(callee, (coroutine,stack))
                         elif callable(callee):
                             # Requested an asynchronous call
                             self._make_async_call(callee, coroutine, stack)
                         else:
                             # Raise a TypeError in the current coroutine
                             self.schedule(coroutine, stack,
                                  TypeError, "Illegal argument to yield"
                             )

                 # Add the new pseudothread to the execution queue
                 self.queue.append(pseudothread)

             def _make_async_call(self, blocking_call, caller, stack):
                 # Assume @threaded decorator takes care of
                 #   - returning a function with a call method which
                 #     kick starts the function execution and returns
                 #     a Future object to give access to the result.
                 #   - farming call out to a physical thread pool
                 #   - keeping the Thread object executing the async
                 #     call alive after this function exits
                 @threaded
                 def async_call():
                     try:
                         result = blocking_call()
                     except:
                         # send the error back to the caller
                         self.schedule(
                             caller, stack, *sys.exc_info()
                         )
                     else:
                         # Send the result back to the caller
                         self.schedule(caller, stack, result)

                 # Start the asynchronous call
                 async_call()


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com