[Tutor] RE: HTTPLib and POST problem ["100 continue" problem redux?]

[Danny Yoo]

> The same is true about PERL( LWP module). Does it mean PERL is
> better??
>
> And as an example you can check 
> http://www.globaltradeweb.com/default.asp?w=fbjm
> 
> You will not be probably post successfully if you send 
> form like
>  /cgi-bin/script.cgi 


As soon as I heard the word "PERL" and "better", I knew I had to step
in.  *grin*


There's actually a very strange problem here: apparently, 'httplib' isn't
handling the '100 continue' server response gracefully.  Your server is
telling httplib that it's ok to pass more headers, but apparently, httplib
isn't prepared for this!

I did some research, and found a kludgy hack to make things work.

    http://mail.python.org/pipermail/python-list/2000-December/023204.html

The following source code should do what you want:

###
import httplib, urlparse

def doPost(url):
    """An simplified interface to do an HTTP post.  Just pass an URL."""
    scheme, location, path, parameters, query, fragment = \
             urlparse.urlparse(url)
    headers = {"Content-type": "application/x-www-form-urlencoded",
               "Accept": "text/plain"}
    conn =  httplib.HTTPConnection(location)
    conn.request("POST", path, query, headers)
    response = safe_getresponse(conn)
    data = response.read()
    conn.close()
    return data


def safe_getresponse(conn):
    ## This is an ugly kludge!  We need to talk with the
    ## comp.lang.python folks!  See:
    ## http://mail.python.org/pipermail/
    ##     python-list/2000-December/023204.html
    ## for a reference to this fix.
    while 1:
        response = conn.getresponse()
        if response.status != 100:
            break
        conn._HTTPConnection__state = httplib._CS_REQ_SENT
        conn._HTTPConnection__response = None
    return response

if __name__ == '__main__':
    url = "http://www.globaltradeweb.com/default.asp?w=fbjm"
    print doPost(url)
###



The code above (crossing my fingers) should let the POST finally work for
you.  I have to admit, thought, that I feel very naughty about this code.  
I feel this is a bug in httplib; does anyone know more about this?