[Web-SIG] Server-side async API implementation sketches

Mon Jan 10 16:12:02 CET 2011

At 05:06 PM 1/9/2011 -0800, Alice BevanMcGregor wrote:
>On 2011-01-09 09:03:38 -0800, P.J. Eby said:
>>Hm.  I'm not sure if I like that.  The typical app developer really 
>>shouldn't be yielding multiple body strings in the first place.
>
>Wait; what?  So you want the app developer to load a 40MB talkcast 
>MP3 into memory before sending it?

Statistically speaking, the "typical app" is producing a web page, 
made of HTML and severely limited in size by the short attention span 
of the human user reading it.  ;-)

Obviously, the spec should allow and support streaming.

>   You want to completely eliminate the ability to stream an HTML 
> page to the client in chunks (e.g. <head> block, headers + search 
> box, search results, advertisements, footer -- the exact thing 
> Google does with every search result)?  That sounds like 
> artificially restricting application developers, to me.

First, I don't want to eliminate it.   Second, Google is hardly the 
"typical app developer".  If you need the capability, it'll still be there.

>>In your approach, the above samples have to be rewritten as:
>>      return app(environ)
>>[snip]
>
>My code does not use return.  At all.  Only yield.

If you return the calling of a generator, then you pass the original 
generator through to the caller, and it is the equivalent of writing 
a loop in place that iterates over the subgenerator, only without the 
additional complexity of needing to send/throw.

>>The above middleware pattern works with the sketches I gaveon the 
>>PEAK wiki, and I've now updated the wiki to include an exampleapp 
>>and middleware for clarity.
>
>I'll need to re-read the code on your wiki; I find it incredibly 
>difficult to grok, however, you can help me out a bit by answering a 
>few questions about it: How does middleware trap exceptions raised 
>by the application.

With try/except around the "yield app(environ)" call (main app run), 
or with try/except around the "yield body_iter" call (body iterator run).

>  (Specifically how does the server pass the buck with 
> exceptions?  And how does the exception get to the application to 
> bubble out towards the server, through middleware, as it does now?)

All that is in the Coroutine class, which is a generator-based "green 
thread" implementation.

Remember how you were saying that your sketch would benefit from PEP 380?

The Coroutine class is a pure-Python implementation of PEP 380, minus 
the syntactic sugar.  It turns "yield" into "yield from" whenever the 
value you yield is itself a geniter.

So, if you pretend that "yield app(environ)" and "yield body_iter" 
are actually "yield from"s instead, then the mechanics should become clearer.

Coroutine runs a generator by sending or throwing into it.  It then 
takes the result (either a value or an exception) and decides where 
to send that.  If it's an object with send/throw methods, it pushes 
it on the stack, and passes None into it to start it running, thereby 
"calling" the subgenerator.  If it's an exception or a return value 
(e.g. StopIteration(value=None)), it pops the stack and propagates 
the exception or return value to calling generator.

If it's a future or some other object the server cares about, then 
the server can pause the coroutine (by returning 'routine.PAUSE' when 
the coroutine asks it what to do).

Coroutine accepts a trampoline function and a completion callback as 
parameters: the trampoline function inspects a value yielded by a 
generator and then tells the coroutine whether it should PAUSE, CALL, 
RETURN, RESUME, or RAISE in response to that particular 
yield.  RESUME is used for synchronous replies, where the yield 
returns immediately.  RETURN means pop the current generator off the 
stack and return a value to the calling generator.  RAISE raises an 
error immediately in the top-of-stack generator.  CALL pushes a 
geniter on the stack.

IOW, the Coroutine class lets you write servers with just a little 
glue code to tell it how you want the control to flow.  It's actually 
entirely independent of WSGI or any particular WSGI protocol...  I'm 
thinking that I should probably wrap it up into a PyPI package with 
some docs and tests, though I'm not sure when I'd get around to it.

(Heck, it's the sort of thing that probably ought to be in the stdlib 
-- certainly PEP 380 can be implemented in terms of it.)

Anyway, both the sync and async server examples have trampolines that 
detect futures and process them accordingly.  If you yield to a 
future, you get back its result -- either a value or an exception at 
the point where you yielded it.  You don't have to explicitly call 
.result() (in fact, you *can't*); it's already been called before 
control gets back to the place that yielded it.

IOW, in my sketch, yielding to a future looks like this:

     data = yield submit(wsgi_input.read, 4096)

without the '.result()' on the end.