[Web-SIG] WSGI thread affinity/interleaving

[Phillip J. Eby]

At 11:59 AM 12/19/2005 -0800, Robert Brewer wrote:
>Couldn't someone write a piece of WSGI middleware that takes requests
>from an async server and dispatches them to a pool of Queues? The
>consumer side of the Queue would then call the WSGI app with the same
>thread each time for a given request, but the async-server side would be
>free to create new requests and fetch results from different threads.
>Sort of an async-to-threaded bridge. I would think, even if you chose
>not to build that into your WSGI wrapper, that it would be generic
>enough to be quite useful for any async server + threaded app. I'll
>refrain from any predictions about performance, however... ;)

This was Jim Fulton's suggestion, and it's beginning to makes more 
sense.  :)  Unfortunately I don't think there's a reasonable way to 
integrate it with the host server's threadpool (e.g. the Twisted threadpool).

We should keep an eye, however, on the fact that the vast majority of WSGI 
apps' requests can and should be handled in a single synchronous 
iteration.  Multiple iterations are primarily useful for large files, and 
streaming/push applications.  These are the *only* reason the spec allows 
multiple writes or iterations.   Applications are supposed to do their own 
buffering in all other cases, to minimize the number of blocks shuffled up 
and down the middleware chain.

That being the case, the simplest way to ensure thread affinity in Twisted 
is to just farm out the entire processing of a given request to a 
reactor.callInThread().  The only applications for which this is not 
suitable will be large files and streaming/push, which will tie up threads 
that they probably shouldn't.  To handle those use cases, a customized 
threadpool mechanism would be needed, wherein each thread would have an 
event loop going over the currently active iterators and adding new ones 
from a master request queue whenever the thread-local queue dropped below a 
threshold.