[Web-SIG] wsgiref questions

[Guido van Rossum]

On 12/20/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 12:06 PM 12/20/2006 -0800, Guido van Rossum wrote:
> >On 12/20/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> >>At 10:12 AM 12/20/2006 -0800, Guido van Rossum wrote:
> >> >We're struggling to use wsgiref behind some Googlish infrastructure,
> >> >and one of the issues we find is that there is code that forbids
> >> >hop-by-hop headers. Unfortunately we have a reason to send one
> >> >(Transfer-encoding).
> >>
> >>Are you sure it's not a Content-Encoding?
> >
> >No, "chunked" is definitely a transfer encoding.
> >
> >> >  What's the reason behind their prohibition, and
> >> >what would be the right way to do this?
> >>
> >>See the thread here for the original rationale:
> >>
> >>http://mail.python.org/pipermail/web-sig/2004-September/000879.html
> >
> >Well, we're using wsgiref's simple_server which only speaks HTTP/1.0,
> >but we really need to speak HTTP/1.1 and use chunked. Where do you
> >recommend we put the chunking instead?
>
> It's expected that an HTTP/1.1 server would handle chunking in the case
> where a WSGI app yields individual blocks, assuming that the client
> supports chunking and the server chooses to do it that way.  The way the
> spec is written, the server *must* ensure that each yielded chunk is
> transmitted to the client before the application is asked for the next
> chunk.  However, it's the server's choice as to whether the chunks will
> just be streamed or transmitted using chunked encoding.
>
> wsgiref does have this comment at one point in the code (wsgiref.handlers),
> btw:
>
>      # XXX Try for chunked encoding if origin server and client is 1.1
>
> This is the place where you could try to add support for doing this.  I.e.,
> that's the place where you'd add the Transfer-Encoding header, after
> checking for client support.

Yeah, but we'd like not to edit wsgiref, so our code won't depend on a
hacked version of it...

> By the way, if your goal is just to ensure that data is written to the
> client as its yielded by the application, that can be done without needing
> the chunked encoding.  My understanding is that all chunked encoding is
> good for is sending data of unknown length over a persistent
> connection.  simple_server doesn't support persistent connections (and WSGI
> doesn't have "connections"), so chunking is of use only if you have some
> crazy kind of client that needs the chunking for some reason known only to
> its developers.  :)

And guess what, that's exactly the situation we're in.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)