[Web-SIG] httplib and 100-continue support

[Graham Dumpleton]

On 26 May 2010 05:58, Deron Meranda <deron.meranda at gmail.com> wrote:
> I'm trying to use httplib to make a PUT request.  I'd like to
> be able to correctly use the 100-continue expectation
> handshake as HTTP/1.1 recommends, such that I wait for
> a 100 status code back before I send the body.  The only
> examples I've found immediately send the request body
> without waiting for a 100 status return.
>
>
> I'd like to do something similar to:
>
> -----
> conn = httplib.HTTPConnection( ... )
> conn.putrequest( "PUT", url )
> if data_length is None:
>    conn.putheader( "Transfer-Encoding", "chunked" )
> else:
>    conn.putheader( "Content-Length", str(data_length) )
> conn.putheader( "Expect", "100-continue" )
> conn.endheaders()
>
> # Wait for any of: timeout, 100, or other error status
>
> if timeout or got_100:
>    # Start sending data using conn.send() method
> -----
>
> There are two problems trying to do this:
>
> 1. The endheaders() doesn't actually send any data.
> You have to do a conn.send( "" ) with an empty string
> for the headers to get to the server, which is needed
> before it can then send back a 100 (or other status code).
>
> 2. There's no way to wait for a 100 response, or wait for
> a response with a timeout -- both of which are needed
> to do the 100-continue logic of HTTP/1.1.
>
> Solving #1 is easy, as long as you know to do it.
>
> Solving #2 appears to require changes to the httplib
> library.  Something along the lines of adding additional
> optional arguments to the getresponse() method, as in:
>
> resp = conn.getresponse( ignore_continue=False, timeout=5 )
>
> if resp is None:
>   # timeout occured
> elif resp.status == httplib.CONTINUE:
>   # got 100 continue
> elif resp.status == httplib.EXPECTATION_FAILED:
>   # server doesn't understand 100-continue
> elif resp.status == httplib.REQUEST_ENTITY_TOO_LARGE:
>   # message too large for server
> elif resp.status == httplib.LENGTH_REQUIRED:
>   # can't do chunked encoding, etc.
>
>
> Is there in fact some way to do this that I'm missing, and if
> not is this something that makes sense to try to fix in
> httplib?

Bigger problem may be to find a server which actually supports
100-continue in the way one would expect it to work.

If you want to upload to a Python web application your options are
possibly quite limited.

In the Apache space, the following definitely work:

* mod_wsgi embedded mode

And the following will not work as desired:

* mod_wsgi daemon mode
* CGI
* FASTCGI
* SCGI
* AJP
* uWSGI ???
* mod_proxy to backend

The problem with the latter is that they all start reading request
content immediately and start sending it through to the backend
process. This has side effect of triggering the 100 status straight
away before the Python web application does anything and so client
starts sending data.

Thus there is no chance with the latter of the Python web application
not reading data and returning an error response before the client is
told it is okay to send the data.

If you use a front end proxy like nginx in front of Apache, even if
using mod_wsgi embedded mode, you also might have problems also as not
sure that implements proper end to end 100-continue processing and
instead also just starts reading client data straight away with side
of effect of generating 100 status.

Graham