[Twisted-web] Re: [Web-SIG] WSGI woes
Phillip J. Eby
pje at telecommunity.com
Thu Sep 16 08:37:06 CEST 2004
At 01:13 AM 9/16/04 -0400, Donovan Preston wrote:
>On Sep 15, 2004, at 7:12 PM, Phillip J. Eby wrote:
>>At 06:48 PM 9/15/04 -0400, Peter Hunt wrote:
>>>It looks like WSGI is not well received over at twisted.web.
>>Excerpting from that post:
>>"""The WSGI spec is unsuitable for use with asynchronous servers and
>>applications. Basically, once the application callable returns, the
>>server (or "gateway" as wsgi calls it) must consider the page finished
>>This is incorrect.
>As I said in my original post, I hadn't mentioned anything about this
>yet because I didn't have a solution or proposal to fix the problem,
>which I maintain remains.
Reading the rest of your post, I see that you are actually addressing the
issue of asynchronous *applications*, and I have only been addressing
asynchronous *servers* in the spec to date. (Technically "half-async"
servers, since to be properly portable, a WSGI server *must* support
synchronous applications, and therefore an async WSGI server must have a
thread pool for running applications, even if it contains only one thread.)
However, I'm not certain that it's actually possible to support *portable*
asynchronous applications under WSGI, since such asynchrony requires
additional support such as an event loop service. As a practical matter,
asynchronous applications today require a toolset such as Twisted or
peak.events in addition to the web server, and I don't really know of a way
to make such applications portable across web servers, since the web server
might use a different toolset that insists on having its own event
loop. Or it might be like mod_python or CGI, and not really have any
meaningful way to create an event loop: it could be utterly synchronous in
nature and impossible to make otherwise.
Thus, as a practical matter, applications that make use of asynchronous I/O
*may* be effectively outside WSGI's scope, if they have no real chance of
portability. As I once said on the Web-SIG, the idea of WSGI is more aimed
at allowing non-Twisted apps to run under a Twisted web server, than at
allowing Twisted applications to run under other web servers! The latter,
obviously, is much more ambitious than the former.
But I'm happy to nonetheless explore whether there is any way to support
such applications without unduly complicating middleware. I don't expect
it would complicate servers much, but middleware can be quite difficult,
because middleware currently isn't even required to return when the
application does! It's not recommended, but a middleware component can sit
there and iterate over the return value and call its parent's write()
method all it wants. In the presence of this kind of behavior, there isn't
any real way to guarantee that a thread isn't going to be tied up with
processing. But realistically, that's what an async server's thread pool
Anyway, as you'll see below, WSGI can actually run async apps with minimal
blocking even without any modifications to the spec, and with
server-specific extensions you can eliminate *all* the blocking, as long as
middleware doesn't do anything pathological. In practice, of course, I
think the spec *should* be updated so that middleware is prohibited from
interfering with the control flow, and I'll give some thought as to how
that should be phrased.
>According to the spec, """The application object must return an
>iterable yielding strings.""" Whether the application callable calls
>write before returning or yields strings to generate content, the
>effect is the same -- there is no way for the application callable to
>say "Wait, hang on a second, I'm not ready to generate more content
>yet. I'll tell you when I am." This means the only way the application
>can pause for network activity is by blocking.
That is correct. The application must block for such activities. However,
as a practical matter, this isn't a problem for e.g. database access, since
using Twisted's adbapi would still tie up *some* thread with the exact same
blocking I/O, so there's actually no loss in simply doing unadorned DBAPI
access from within the application.
> For example, a page
>which performed an XML-RPC call and transformed the output into HTML
>would be required to perform the XML-RPC call synchronously. Or a page
>which initiated a telnet session and streamed the results into a web
>page would be required to perform reads on the socket synchronously.
Technically, it could perform these tasks asynchronously, as long as the
data were queued such that the application's return iterable simply
retrieved results from the queue. However, this would naturally block
whenever the client was ready for I/O, but no results were available yet.
However, an asynchronous server isn't going to sit there in a loop calling
next()! Presumably, it's going to wait until the previous string gets sent
to the client, before calling next() again. And, it's presumably going to
round-robin the active iterables through the threadpool, so that it doesn't
keep blocking on iterables that aren't likely to have any data to produce
Yes, this arrangement can still block threads sometimes, if there are only
a few iterables active and they are waiting for some very slow async
I/O. But the frequency of such blockages can be further reduced with a
couple of extensions. Suppose there was an 'environ["async.sleep"]' and
'environ["async.wake"]'. A call to 'sleep' would mean, "don't bother
iterating over me again until you get a 'wake' call".
This *still* wouldn't prevent some item of middleware from hogging a thread
in the threadpool, but I suppose you could actually make the 'sleep'
function sit in a loop and run active iterables' next() methods until one
of the suspended iterables in the current thread "wakes", at which point it
would return control to whatever iterable it was called from. Or, if you
want to use Greenlets, you can always return control directly to the
iterable that needs to "wake up".
Anyway, my point here is that it's possible to get a pretty decent setup
for async applications, without any need to actually modify the base WSGI
spec. And, if you add some optional extensions, you can get an even
smoother setup for async I/O.
Finally, I'm open to trying to define the 'sleep/wake' facilities as
"standard options" in WSGI, as well as clarifying the middleware control
flow to support this better.
>The server or gateway, by calling next(), is assuming that the call
>will yield a string value, and only a string value.
The spec doesn't rule out empty strings, however, which would be the
natural way to indicate that no data is available. So, the protocol in an
async app's iterator would be something like:
if 'async.wake' in environ:
# We should only get to this line once environ['async.wake']
has been called
# delay an exponentially increasing period if queue is still
If middleware is required to match the control flow of the application it
wraps (e.g. write()=>write(), yield=>yield), then this would result in
complete non-blockingness when the server supports the 'async' extensions.
Of course, a blocking delay *is* required when running in a server that
doesn't support the async extensions, but that's unavoidable in that
case. (Technically, you might be better off just doing synchronous I/O if
you're being run in a synchronous server, but that's of course optional.)
>"""The application object must return an iterable yielding strings or
>objects implementing the following interface:
> '''Add 'callable' to the list of callables to be invoked when a
> is available. Callable should take a single argument, which will
> be a
>The application object must invoke the callable passed to addCallback,
>passing a string which will be written to the request.
>This places additional burdens upon implementors of WSGI servers or
And a near-intolerable burden on middleware, which would have to have a way
to "pass through" this facility. It would be much better to limit the
pass-through requirements to covering write and yield, rather than
requiring middleware to implement addCallback facilities as well.
>Finally, we come to the task of implementing a server or gateway which
>can asynchronously support either asynchronous or blocking
>applications. Since there is no way for the server or gateway to know
>whether the application object it is about to invoke will block,
>starving the main loop and preventing network activity from being
>serviced, it must invoke all applications in a new thread or process.
But *some* thread is going to be working on it, and this is true whether
you use a thread pool or the server is purely synchronous. And, because a
WSGI server *must* support synchronous applications, it *must* have some
thread available that is amenable to blocking.
Of course "new" threads are not required. I assume that in the case of
Twisted, something like reactor.deferToThread() will be used to wrap a WSGI
application's initial invocation, and each individual 'next()' call.
More information about the Web-SIG