using urllib or httplib to post with ENCTYPE=mulitpart/form-data

[John J. Lee]

Kevin Carlson <nskhcarlso at bellsouth.net> writes:

> John J. Lee wrote:
> > Kevin Carlson <nskhcarlso at bellsouth.net> writes:
> > [...]
> >
> 
> > No reason why not, but IMHO perverse.  :-)
> >
> 
> I agree, it is easier to use urllib, but in this case I have to post

Easier still to use urllib2.  AFAICS, urllib2 is better than urllib,
period.  I don't think it makes sense to use urllib for new code --
perhaps it should be deprecated.

I suspect there's a perception amongst Python web-scrapers that using
urllib2 means you're stuck when you get down to the nitty gritty of
HTTP headers, but that's not true.  Not sure how true that is of
urllib, because I've rarely used it.


> data in multipart/form-data which means MIME.  I can build the proper
> request and send it using httplib, but haven't been able to figure out
> how to do this with urllib.

Like I said, ClientForm (0.1.2a) should do this automatically (though
I admit the multipart/form-data stuff is dodgy ATM).  Regardless of
presence or absence of file upload controls.

response = urllib2.urlopen("http://www.example.com/")
forms = ClientForm.ParseResponse(response)
form = forms[0]
response2 = urllib2.urlopen(form.click())


> If I create the MIME data as 'data', and them post the request using
> urllib.request(URL, data), it doesn't seem to post the form
> correctly. If I do the same with httplib it works fine.  Am I missing
> something?

The third argument to the urllib2.Request constructor (or the
urllib2.Request.add_header method)?

request = urllib2.Request(url, data, headers)
response = urllib2.urlopen(request)
print response.read()


Obviously though, I wouldn't do that myself -- I'd use ClientForm :-)

BTW, note that urllib2 in 2.3b1 is broken (whose fault could that be,
I wonder?).


John