[Web-SIG] Most WSGI servers close connections to early.

[Robert Brewer]

Benoit Chesneau wrote:
> On Wed, Sep 22, 2010 at 5:34 PM, Robert Brewer <fumanchu at aminus.org>
> wrote:
> > However, the caveat requires a caveat: servers must still be able to
> protect themselves from malicious clients. In practice, that means
> allowing servers to close the connection without reading the entire
> request body if a certain number of bytes is exceeded.
>
> I don't see how it could be the responsability of the server. Can you
> develop a little ? The server shouldn't interfere in the HTTP request
> imo.

Well since the "origin server" is the only component in the architecture
that's *actually* having an HTTP conversation with the client, calling
it "interference" seems a bit skewed. ;) RFC 2616 8.2.3 says:

"If an origin server receives a request that does not include an
Expect request-header field with the "100-continue" expectation,
the request includes a request body, and the server responds
with a final status code before reading the entire request body
from the transport connection, then the server SHOULD NOT close
the transport connection until it has read the entire request,
or until the client closes the connection. Otherwise, the client
might not reliably receive the response message. However, this
requirement is not be construed as preventing a server from
defending itself against denial-of-service attacks, or from
badly broken client implementations."

The way CherryPy implements this is to wrap the socket file before
handing it to wsgi.input. That wrapper understands Content-Length (and
another understands Transfer-Encoding), and won't allow any component
that calls wsgi.input.read(n) to read past the Content-Length limit.
[This also allows components to call read() without a size argument yet
not timeout on the socket, as specified in recent proposals.]

The server can be configured to have a maximum number of bytes it will
allow to be read--if Content-Length exceeds that number, the server
immediately responds with 413 Request Entity Too Large. It doesn't read
the rest of the request entity, because it's too big and could cause a
DoS. If clients can't read the response because they're still blocked
sending a request that's too big, there's not really any way to get
around that if the client didn't send an Expect request header.

If the Content-Length is not too large, and the application returns
(normally or exceptionally), and the wrapper has not recorded that the
bytes read equals the Content-Length, then the server will consume the
remaining bytes and throw them away before sending the response headers.

I just noticed it doesn't do that if it's going to close the conn. Not
sure why. Maybe it should.


Robert Brewer
fumanchu at aminus.org