[Web-SIG] [PEP 444] Future- and Generator-Based Async Idea

[P.J. Eby]

At 03:26 AM 1/8/2011 -0800, Alice Bevan­McGregor wrote:
>Warning: this assumes we're running on bizzaro-world PEP 444 that 
>mandates applications are generators.  Please do not dismiss this 
>idea out of hand but give it a good look and maybe some feedback.  ;)

First-glance feedback: I'm impressed.  You may have something going 
here after all.  I just wish you'd sent this sooner.  ;-)

I can easily see why I didn't think of this myself: I hadn't shifted 
my thinking to accomodate for two important changes in the Python 
environment since the first WSGI spec, circa 2003-04:

1. Coroutines and decorators are ubiquitous and non-intrusive
2. WSGI has stdlib support, and in any event it is much easier to 
rely on non-stdlib packages

My major concern about the approach is still that it requires a fair 
amount of overhead on the part of both app developers and middleware 
developers, even if that overhead mostly consists of importing and 
decorating.  (More below.)


>The second middleware demonstration (using a decorator) makes 
>middleware look a lot more like an application: yielding futures, or 
>a response, with the addition of yielding an application callable 
>not explored in the first (long, but trivial) example.  I believe 
>this should cover 99% of middleware use cases, including interactive 
>debugging, request routing, etc. and the syntax isn't too bad, if 
>you don't mind standardized decorators.

If we assume that the implementation would be in a wsgi2ref for 
Python 3.3 and distributed standalone for 2.x, I think we can make 
something work.  (In the sense of practical to implement, not 
necessarily *desirable*.)

One of my goals is that it should be possible to write "async-naive" 
applications and middleware, so that people who don't care about 
async can ignore it.

On the application side, this is easy: a trivial decorator suffices 
to translate a return into a yield.

For middleware, it's not quite as simple, unless you have a pure 
ingress or egress filter, since you can't simply "call" the 
application.  However, a "context manager"-like pattern applies, 
wherein you could simply yield to calling a wrapped version of the application.

Hm.  This seems to pretty much generalize to a standard 
coroutine/trampoline pattern, where the server provides the 
trampoline, and can provide APIs in the environ to create waitable 
objects that can be yielded upward.

Actually, this is kind of like what I really wanted the futures PEP 
to be about.  And it also preserves composability nicely.

In fact, it doesn't actually need any middleware decorators, if the 
server provides the trampoline.

We would leave your "my_awesome_application" example intact (possibly 
apart from having a friendlier API for reading from wsgi.input), but 
change my_middleware as follows:

    def my_middleware(app):
        def wrapper(environ):
            # pre-response code here
            response = yield app(environ)
            # post-response code here
            yield altered_response
        return wrapper

That's it.  No decorators, no nothing.

The server-level trampoline is then just a function that looks 
something like this:

     def app_trampoline(coroutine, yielded):
         if [yielded is a future of some sort]:
             [arrange to invoke 'coroutine(result)' upon completion]
             [arrange to inovke 'coroutine(None, exc_info)' upon error]
             return "pause"
         elif [yielded is a response]:
             return "return"
         elif [yielded has send/throw methods]:
             return "call"  # tell the coroutine to call it
         else:
             raise TypeError

The trampoline function is used with a coroutine class like this:

     class Coroutine:

         def __init__(self, iterator, trampoline, callback):
             self.stack = [iterator]
             self.trampoline = trampoline
             self()

         def __call__(self, value=None, exc_info=()):
             stack = self.stack
             while stack:
                 try:
                     it = stack[-1]
                     if exc_info:
                         try:
                             rv = it.throw(*exc_info)
                         finally:
                             exc_info = ()
                     else:
                         rv = it.send(value)
                 except BaseException:
                     value = None
                     exc_info = sys.exc_info()
                     if exc_info[0] is StopIteration:
                         # pass return value up the stack
                         value, = exc_info[1].args or (None,)
                         exc_info = ()   # but not the error
                     stack.pop()
                 else:
                     switch = self.trampoline(self, rv)
                     if switch=="pause":
                         return
                     elif switch=="call":
                         stack.append(rv)  # Call subgenerator
                         value, exc_info = None, ()
                     elif switch=="return":
                         value, exc_info = rv, ()
                         stack.pop()

             # Coroutine is entirely finished
             self.callback(value)

And run by simply calling:

     Coroutine(app(environ), app_trampoline, process_response)

Where process_response() is a function receiving a three-tuple to 
process the actual result.

That's basically it.  The Coroutine class is 
server/framework-independent; the minimal trampoline function is the 
part the server author has to write.

The body iterator can follow a similar protocol, but the trampoline 
function is different:

     def body_trampoline(coroutine, yielded):
         if type(yielded) is bytes:
             if len(coroutine.stack)==1:  # only accept from 
outermost middleware
                 [send the bytes out]
                 [arrange to invoke coroutine() when send is completed]
                 return "pause"
             else:
                 return "return"
         if [yielded is a future of some sort]:
             [arrange to invoke 'coroutine(result)' upon completion]
             [arrange to inovke 'coroutine(None, exc_info)' upon error]
             return "pause"
         elif [yielded has send/throw methods]:
             return "call"  # tell the coroutine to call it
         else:
             raise TypeError

So, part of the server's "process_response" callback would look like:

     Coroutine(body_iter, body_trampoline, finish_response)


You can then implement response-processing middleware like this:

     def latinize_body(body_iter):
         while True:
             chunk = yield body_iter
             if chunk is None:
                 break
             else:
                 yield piglatin(yield body_iter)

     def piglatin(app):
         def wrapper(environ):
             s, h, b = yield app(environ)
             if [suitable for processing]:
                 yield s, h, latinize_body(b)
             else:
                 yield s, h, b  # skip body processing


My overall impression is still that there's something worth 
considering here, but there is still some ugly mental overheads 
involved for body-processing middleware, if we want to support 
pausing during the body iteration.  The latinize_body function above 
isn't exactly intuitively obvious, compared to a for loop, and it 
can't be replaced by one without using greenlets.

On the plus side, it can actually all be done without any decorators at all.

(The next interesting challenge would be to integrate this with 
Graham's proposal for adding cleanup handlers...)