[Web-SIG] ngx.poll extension (was Re: Are you going to convert Pylons code into Python 3000?)

Brian Smith brian at briansmith.org
Thu Mar 6 16:08:18 CET 2008


Manlio Perillo wrote:
> Brian Smith ha scritto:
> > Manlio Perillo wrote:
> >> Fine with me but there is a *big* problem.
> >>
> >> WSGI 2.0 "breaks" support for asynchronous applications (since you 
> >> can no more send headers in the app iter).
> > 
> > WSGI 1.0 doesn't guarentee that all asynchronous applications will 
> > work either, because it allows the WSGI gateway to wait for 
> and buffer 
> > all the input from the client before even calling the 
> application callable.
> > And, it doesn't provide a way to read an indefinite stream of input 
> > from the client, which is also problematic.
> > 
> > Anyway, please post a small example of a program that fails to work 
> > because of these proposed changes for WSGI 2.0.
> > 
> > Thanks,
> > Brian
> > 
> 
> 
> Attached there are two working examples (I have not committed 
> it yet, because I'm still testing - there are some problems 
> that I need to solve).

I looked at your examples and now I understand better what you are
trying to do. I think what you are trying to do is reasonable but it
isn't something that is supported even by WSGI 1.0. It happens to work
efficiently for your particular gateway, but that isn't what WSGI is
about. In fact, any WSGI application that doesn't run correctly with an
arbitrary WSGI gateway (assuming no bugs in any gateway) isn't a WSGI
application at all.

It seems that the problem with your examples is not that they won't work
with WSGI 2.0. Rather, the problem is that the applications block too
long. The application will still work correctly, but will not be
efficient when run in nginx's mod_wsgi. However, that isn't a problem
with the specification or with the application; it is a problem with
nginx's mod_wsgi. I hate reading about the "Pythonic way" of doing
things, but writing a WSGI application so that it doesn't block too much
or too long is simply not Pythonic. The WSGI gateway needs to abstract
away those concerns so that they aren't an issue. Otherwise, the gateway
will only be useful for specialized applications designed to run well on
that particular gateway. Such specialized applications might as well use
specialized (gateway-specific) APIs, if they have to be designed
specifically for a particular gateway anyway.

Further, it is impossible to write a good HTTP proxy with WSGI. The
control over threading, blocking, I/O, and buffer management is just not
there in WSGI. In order to support efficient implementations of such
things, WSGI would have to become so low-level that it would become
pointless--it would be exposing an interface that is so low-level that
it wouldn't even be cross-platform. It wouldn't abstract away anything.

At the same time, the current WSGI 2.0 proposal abstracts too much. It
is good for applications that are written directly on top of the
gateway, and for simple middleware. But, it is not appropriate for a
serious framework to be built on. It is wrong to think that the same
interface is suitable for frameworks, middleware developers, and
application developers. I would rather see WSGI 2.0 become a much
lower-level framework that works at the buffer level (not strings), with
the ability to do non-blocking reads from wsgi.input, and the ability to
let the WSGI gateway do buffering in a sane and efficient manner
(there's no reason for the application to do a bunch of string joins
when the gateway could just send all the pieces in a single writev()).
Some control over blocking, HTTP chunked encoding, etc. could be
included as well. The current suggestions for WSGI 2.0 would then just
be a sample framework layered on top of this low-level interface, for
developers that don't want to use a big framework like DJango or Pylons.
But, the big frameworks and middleware would use the low-level interface
to run efficiently. 

- Brian



More information about the Web-SIG mailing list